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The only person utho is educated is the one who has learned how to learn and change. — Carl Rogers 

In a diverse portfolio of curricular and programmatic options in colleges and universities, global 
education has become highly prominent over the past 50 years (O'Meara, Mehlinger, & Newman, 2001; De 
Wit & Merkx, 2012). To take one of any number of indices in this regard, the Institute of International 
Education (2014) reported that a record number of global students, 886,052, studied at U.S. institutions in 
2013-14 (up 8.1 percent over the previous year) and 289,408 American students studied abroad in 2012-13 
(an increase of 2.1 percent over the previous year) (see also International Association of Universities, 2010; 
Knight, 2006). At a complementary level, institutions of higher education now are interested in “the 
internationalization of research, offering dual degrees with foreign partners, establishing a branch of campuses 
abroad,” among other areas of emphasis (Marmolejo, 2011). Duru and Poyrazli (2007) contend that these 
sorts of trends demonstrate the important role that colleges and universities play in training workers for entry 
into the global marketplace. Likewise, Green (2013) notes that “higher education institutions in the United 
States are increasingly using the language of ‘global citizenship’ to describe the skills and habits they seek to 
cultivate in their students” (p. 52). Against this introductory backdrop, we begin this article with an overview 
of global education in the United States and a review of assessment instruments and approaches. We then will 
present 15 implications from the Forum-BEVI Project, a multi-year, multi-site assessment initiative that 
examined the processes and outcomes of international, multicultural, and transformative learning. 1 


1 With permission from Springer Publishing, this article for Frontiers is adapted from Wandschneider, E., Pysarchik, 
D.T., Sternberger, L.G., Ma, W, Acheson, K., Baltensperger, B., Good, R.T., Brubaker, B., Baldwin, T., Nishitani, H., 
Wang, E,Reisweber, J., Hart, V. (in press). The Forum BEVI Project: Applications and implications for international, 
multicultural, and transformative learning. In C.N. Shealy (Ed.), Making sense of beliefs and values. New York, NY: 
Springer Publishing. See http://www.springerpub.com/making-sense-of-beliefs-and-values.html . 
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Global Education in the United States 

In 2005, the Forum on Education Abroad investigated international education programming in 46 U.S. 
colleges and universities (Forum, 2005). Results suggested that study abroad programs are intimately related 
to their curricula and objectives. Most of the surveyed institutions integrated global education into the 
international curricula of their students; 91 percent of the institutions usually or always counted credits 
earned by students while abroad toward major or minor requirements. Moreover, 77 percent of the 
respondents noted that credit completed abroad usually or always counted towards general education 
requirements. Similar to the curricular changes at many U.S. universities, the composition of matriculating 
students also has changed in the last 50 years. Such students today often have prior international exposure, 
which is related in part to the fact that transportation to and from other countries is much more accessible. 
Furthermore, young adults are exposed to the beliefs and practices of different cultures through television, the 
Internet, and social media. In addition, many students anticipate that cultural competence will be required of 
them upon graduation from college or university at a level that simply was unimaginable 50 years ago. Thus, 
it is not surprising that a survey by the American Council on Education (ACE) found that most students 
studying abroad had prior international experience, with 64 percent of students polled already having lived in 
or traveled to countries outside of the United States (Siaya & Flayward, 2003). 

Today’s students appreciate the importance of grappling with events and issues that are occurring beyond 
their home borders, in part because of an increasing recognition that global and domestic affairs are 
inextricably linked. For example, 65 percent of the students in the 2003 ACE survey concurred that all 
students should be required to take courses covering global topics (Siaya & Flayward, 2003). Moreover, when 
entering the global marketplace, graduating U.S. students who have studied abroad arguably possess an 
advantage over those who have not. Indeed, “students who lack access to cultural capital may not appreciate 
the significance of study abroad experiences specifically and global understanding in general” (Flu, Pazaki, & 
Velander, 2014, p. 73). The advantage of study abroad consists of more than cross-cultural experience, and is 
related directly to the acquisition of knowledge, skills, and attitudes that cannot be acquired without real 
world experience in another culture. And in fact, many global education programs are designed explicitly to 
facilitate the acquisition of greater sensitivity toward people from other cultures (O'Meara, Mehlinger, & 
Newman, 2001). As Ramirez (2013) observes, however, such student outcomes are not preordained simply 
by studying abroad; rather, “the curriculum of these preparation programs needs to intentionally address 
cultural sensitivity” (p. 8). Thus, it is the responsibility of institutions of higher education to “develop 
international curricula that foster a student citizenry with stronger global awareness” (Tarrant, Rubin, & 
Stoner, 2013, p. 2). 


The Assessment of Global Learning 

As the importance of international education continues to ascend in the U.S. and globally, a 
corresponding need has emerged to assess the outcomes and impacts of such learning (Deardorff & Van 
Gaalen, 2012). In this regard, assessment instruments must be chosen in the context of how a given 
institution defines internationalization (Coryell, Durodoye, Wright, Pate, & Nguyen, 2012). Knight (2011) 
observes further that the benefits of global education cannot be measured simply by counting “the number of 
global students, foreign faculty, institutional agreements, cross-border education programs, research projects, 
foreign accreditations, branch campuses, and so on” (p. 15). Such indices may address certain accountability 
requirements, but they don’t reflect the emotional, cognitive, and behavioral impact of globalization. As 
Zimmermann and Neyer (2013) comment in a study on the personality development of sojourners, “hitting 
the road has substantial effects on who we are” (p. 527). Although assessment and education, in general, long 
have been linked in the United States, an emphasis on the assessment of international, multicultural, and 
transformative learning has been a relatively recent phenomenon, with various attempts underway to define 
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and determine appropriate assessment methods of intercultural competencies (e.g., Deardorff, 2006; Fantini, 
2009; Rexeisen, 2012). In this regard, Sternberger, Pysarchik, Yun, and Deardorff (2009) contend that an 
integrated approach to learning assessment - which includes a carefully planned and well-implemented 
assessment protocol that is linked to theory and uses multiple methods - may be of considerable benefit to 
higher education institutions, by helping students develop measurable competencies that are critical to success 
in an increasingly interconnected world. Along these lines, the Georgia Learning Outcomes of Students 
Studying Abroad Research Initiative (Sutton and Rubin, 2004), the Georgetown Consortium Project (Vande 
Berg, Connor-Linton, & Paige, 2009), and the Wabash National Study of Liberal Arts (Salisbury, 2011) are 
but three large, multi-institutional studies regarding the learning outcomes of education abroad. As 
summarized in the project Student Experience in the Research University (SERU), “formal study abroad 
programs ... bring value-added components to students’ intercultural and global competencies that generally 
meet or surpass the outcomes of other international travel opportunities” (Stebleton, Soria, & Cherney, 2013, 
p. 15). From the standpoint of assessment, several large-scale initiatives to develop methods and instruments 
for measuring globalization recently have been pursued in different countries, including, but not limited to, 
the Association of Commonwealth Universities’ Management Benchmarking Program (2012); NAFSA’s 
“Accessing Best Practices in International Education” (2008); the Netherlands’ Organization for International 
Cooperation in Lligher Education’s “Measuring Internationalization” or MINT project (2012); “A Guide to 
Outcomes Assessment in Education Abroad” (2007), from the Forum on Education Abroad; and, the Forum- 
BEVI Project (2015) - the focus of this article - which investigates the processes and outcomes of global, 
intercultural, and transformative learning. 

So how has the field of international education approached the evaluation of learning outcomes? 

Overall, a review of the literature suggests that assessment studies have examined factors such as content 
knowledge concerning a specific culture or knowledge of global affairs, language acquisition, intercultural 
sensitivity, competence and communication, and salient personality variables, among other constructs and 
processes. Recognizing that the relative merits of particular assessment methods depend upon differences in 
program goals or content as well as realities "on the ground" (Poole & Davis, 2006), typical assessment 
methods include tests, portfolios, interviews, reflective essays, surveys, and inventories. Although not 
intended to be exhaustive, it may be helpful to describe a number of these measures and methods in order to 
illustrate their different applications in research and practice (e.g., see Fantini, 2009). 

Some global learning programs are interested in assessing students’ knowledge of issues that reflect global 
awareness and citizenship. For example, the Global Awareness Profile (GAP) is a self-report inventory that 
includes 126 questions based on common knowledge in two main categories: seven geographical areas (i.e., 
Asia, Africa, North America, South America, the Middle East, Europe, and Global) and seven subject areas 
(i.e., environment, culture, politics, geography, religion, socioeconomics, and global) (Corbitt, 2011). 

Language acquisition is another significant component of many global learning programs, which requires 
its own form of assessment. For example, the American Council on the Teaching of Foreign Languages 
(ACTFL) developed proficiency guidelines in 2012 to assess writing and speaking proficiency levels as well as 
oral proficiency, using Oral Proficiency Interviews (OPI) and Simulated Oral Proficiency Interviews (SOPI) 
(ACTFL, 2012). Likewise, the Interagency Language Roundtable (2011) developed a set of skill level 
descriptions by which language proficiency tests may be developed and scored. 

At the same time, there is increasing recognition that such competence involves more than just 
proficiency in reading, writing, listening, and speaking. As Scarino (2010) notes, comprehensive evaluation of 
foreign language learning programs should include cultural aspects of learning, because “in any act of 
learning, people (as young persons and as learners) are engaged in interpreting self (intraculturality) and other 
(interculturality) in diverse contexts of social and cultural interaction” (p. 326). Along these lines, the 
European Language Portfolio (ELP) requires students to assemble three components: 1) a language passport 
that includes basic information about a student’s linguistic identity; 2) a language biography that includes 
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language learning goals, progress, and record of language learning and intercultural experiences; and 3) a 
dossier that contains a selection of a student’s work that is representative of his/her foreign language 
proficiency (Little, 2002). 

Although foreign language and global knowledge are included as part of the content of what is learned, 
much assessment regarding global learning has focused on attitudinal, cognitive, and affective change. As an 
early example of such measurement over 50 years ago, Sampson and Smith (1957) attempted to measure 
“world-mindedness,” which was concerned with the problems of humanity rather than the problems of a 
specific nation or culture. The 32-item World-mindedness Scale assesses eight dimensions: religion, 
immigration, government, economics, patriotism, race, education, and war. In an attempt to update this 
measure and address various psychometric issues, Hett (1993) developed the Global-mindedness Scale, which 
reflects attitudes, beliefs, and behaviors, in order to assess “a worldview in which one sees oneself as connected 
to the world community and feels a sense of responsibility for its members” (p. 143), and consists of five 
dimensions: responsibility, cultural pluralism, efficacy, global-centrism, and interconnectedness (see Clarke, 
Flaherty, Wright, & McMillen, 2009; Kehl & Morris, 2007). Along related lines, Der-Karabetian (1992) 
designed the Cross-cultural World-mindedness Scale, which consists of 26 items that evaluate attitudes toward 
race, religion, immigration, patriotism, economics, war, world government, and global education (Der- 
Karabetian & Metzer, 1993). Likewise, Shaftel and Shaftel (2010) developed the Intercultural Student 
Attitude Survey (ISAS), a 35-item scale created to assess changes in the attitudes and future plans of 
undergraduate students who study abroad. This measure focuses on assessing motivational cultural 
intelligence, which encompasses “the desire to learn about and function effectively in intercultural situations” 
(p. 12). More specifically, ISAS measures six constructs: cross-cultural open-mindedness, ability to manage 
stress, adventurousness, self-confidence, disposition toward a global career, and foreign language study. 

A number of related measures focus on intercultural sensitivity, competence, and communication. For 
example, Chen and Starosta (2000a, 2000b) define intercultural communication competence as an umbrella 
concept comprised of cognitive (intercultural awareness), behavioral (intercultural adroitness), and affective 
(intercultural sensitivity) abilities that impact intercultural communication. Bhawuk and Brislin (1992) argue 
that intercultural effectiveness requires people to be “interested in other cultures ... sensitive enough to notice 
cultural differences, and ... willing to modify their behavior as an indication of respect for the people of other 
cultures” (p. 416). Along these lines, Fantini (2000) helpfully sought to organize these levels of analysis by 
juxtaposing the categories of awareness, attitude, skills, knowledge, and second language proficiency, with four 
developmental levels of competence under each category: educational traveler, sojourner, professional, and 
intercultural/multicultural specialist. 

Based upon the Developmental Model of Intercultural Sensitivity (DMIS) (Bennett, 1986, 1993; 

Bennett & Hammer, 2002), the Intercultural Development Inventory (IDI) is designed to measure one’s 
status and progression through developmental stages, including denial, polarization, minimization, 
acceptance, and adaptation; thus, the stages of DMIS theoretically represent an individual's increasingly 
sophisticated capacity to apprehend and accurately experience cultures that are different from their own (e.g., 
Hammer, Bennett, & Wiseman, 2003; Rexeisen, Anderson, Lawton, & Hubbard, 2008). Such stages may be 
plotted along a continuum representing a person's cultural sensitivity, ranging from monocultural to 
intercultural mindsets (Hammer, 2012). From the standpoint of DMIS, when people become more 
interculturally engaged, they are not just becoming more sensitive to a particular culture; rather, their 
intercultural view and experience will extend to other cultures, including their own. 

Regarding processes of navigating different cultures and contexts, the Cross-Cultural Adaptability 
Inventory (CCAI) assesses an individual’s ability to adapt to different cultures. A 50-item self-report measure, 
the CCAI helps an individual identify his/her strengths and weaknesses within four skill areas of intercultural 
communication and interaction: emotional resilience, flexibility/openness, perceptual acuity, and personal 
autonomy (Kelley & Meyers, 1995; Chang, Yuan, & Chuang, 2013). The International Education Survey 
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(IES) has been used “to assess how an international experience impacts personal and intellectual development” 
(Rexeisen, Anderson, Lawton, & Hubbard, 2008, p. 2). Relatedly, Van der Zee and van Oudenhoven (2000) 
developed the Multicultural Personality Questionnaire, which measures multicultural effectiveness defined as 
“success in the fields of professional effectiveness, personal adjustment, and intercultural interactions” (p. 

293). This measure has five scales: cultural empathy, open-mindedness, emotional stability, social initiative, 
and flexibility. 

As a final example, the Global Perspective Inventory (GPI) and Global Competence Aptitude Assessment 
(GCAA) measure three dimensions: cognitive (knowing and knowledge: “how do I know?”), intrapersonal 
(identity and affect: “who am I?”), and interpersonal (social responsibility and social interaction: “how do I 
relate?”) (see Doyle, 2009). Overall, the GCAA measures dimensions of knowledge, skills, and attitudes as 
well as internal and external readiness to experience different cultures (Braskamp, Braskamp, Merrill, & 
Engberg, 2010). 

While the above overview is not intended to be exhaustive, it should illustrate the types of measures that 
are being used within the field of international education assessment as well as the main foci of each approach. 
To be sure, many additional instruments might have been featured. For example, Fantini (2006) profiles 87 
such measures under the aegis of “Assessment Tools of Intercultural Communicative Competence” (see also 
Fantini, 2009). The breadth of this compendium illustrates both the challenge in selecting the “right” 
assessment measure for a particular purpose as well as an attendant need to specify standards by which 
measures should be evaluated, research conducted, and practice implemented within the broader field of 
international, multicultural, and transformative learning. To provide guidance, the Standards of Good Practice 
for Education Abroad, developed by the Forum on Education Abroad, offers a comprehensive system by which 
the goals and practices of assessment may be evaluated. In this regard, “The Forum Standards are intended to 
be used on an ongoing basis to respond to the practical realities of developing, managing, and assessing 
education abroad programs” (Whalen, 2011, p. 3). Many of the instruments described above - the IDI and 
the GPI, among others - as well as the Beliefs, Events, and Values Inventory (BEVI) described below, 
deliberately attempt to align their approach to measurement with these Forum Standards (e.g., Braskamp, 

Blair, Shealy, & Hammer, 2012). 


International, Multicultural, and Transformative Assessment: 

Key Questions 

The above overview of global learning approaches provides a sense of how scholars and practitioners have 
attempted to measure the impact of international education, broadly defined. In so doing, many questions 
emerge. For example: "How do we determine which constructs are most and least salient?” “Whose questions 
are and are not answered?” “Can we actually measure such complex phenomena in a valid and reliable 
manner?” In addressing such matters, Sternberger, La Brack, and Whalen (2007) observe the following: 

Increasingly, faculty and administrators within the field of study abroad are taking a closer and 
more nuanced look at the experiences of study abroad, considering in more detail the cognitive, 
emotional, and developmental changes that may occur as a result of the experience (p. 78). 

In many ways, the movement to assess multiple dimensions of global learning mirrors what occurred in the 
U.S. education system 30 years ago, when more sophisticated models and measurement techniques were 
introduced into U.S. classrooms. Such innovations allowed for the evaluation, monitoring, and improvement 
of learning over time, while recognizing that students may respond differently to standard instruction in a way 
that is congruent with their learning and psychological profiles (Lerner, 2003). 

Within the field of global education, however, a similar process of understanding and explaining the 
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complexities of the interaction of differences among learners with learning processes and outcomes is a 
relatively recent undertaking. In all likelihood, that is because accounting for learning differences in a 
traditional environment is compounded substantially by the realities of living and learning abroad. While 
such learning is similar in some ways to that which occurs in a traditional classroom, it is qualitatively 
different on another level since study abroad may be experienced and anecdotally referred to as “dramatic” and 
“life-changing” (Doyle, 2009, p. 143), in part due to the “high impact” nature of such learning (e.g. Iyer, 
2013; Kuh, 2008). Thus, students may not experience such learning in the same way or to the same degree. 
At the same time, substantial outcome variability also is apparent: some students report changes that are 
minimal, consisting of increased knowledge of facts regarding the foreign culture they visited, or even negative 
(e.g., “I couldn’t wait to get home”); others emerge fundamentally transformed, no longer experiencing self, 
others, or the larger world as they did prior to the experience (e.g., Cultivating the Globally Sustainable Self, 
2015). Given such variability, then, if we truly wish to understand who learns what and why, and under what 
circumstances, our approach to assessment research and practice must be sophisticated, comprehensive, and 
sustained in order to account for the complex interactions that we - as educators, scholars, and administrators 
- sense in our students but don’t necessarily directly apprehend (e.g., Church, 1982; Reisweber, 2008; 
Sternberger, La Brack, & Whalen, 2007). 

Although there is a recent push to measure the affective and cognitive variables that are associated with 
global learning, assessment research and practice that is designed explicitly to account for such complexity is 
relatively rare. Instead, much of the scholarship in this area focuses on the acquisition of factual information 
or attitudinal change in students as well as the mapping of developmental stages such as orientation, study 
abroad, and re-entry (e.g., Koskinen & Tossavainen, 2003). Such work is very important and highly 
compatible with aspects of what is reported below. However, if we are to understand the processes that 
moderate and mediate change via international, multicultural, and transformative learning, our theoretical 
models and assessment methods must be able to account for such interacting complexities as they occur for 
real human beings in real time in the real world. To take but one example, the disequilibrating experience 
that many students encounter while studying abroad is commonly described as culture shock. As Kohls (2001) 
observed, culture shock results from: 

being cut off from the cultural cues and known patterns with which you are familiar, ... living 
and/or working over an extended period of time in a situation that is ambiguous; ... having your 
own values ... brought into question, which yanks your moral rug out from under you; and ... 
being continually put into a position in which you are expected to function with maximum skill 
and speed but where the rules have not been adequately explained (pp. 93-94). 

Although we concur with the validity of such an observation, the underlying affective, cognitive, 
cultural, contextual, and developmental processes that “cause” an individual to experience “culture shock” to 
varying degrees are not sufficiently illuminated by a descriptive level of analysis alone (Reisweber, 2008). In 
short, despite many helpful and productive advances in this area, the complex variables that mediate and 
moderate global learning processes and outcomes still await precise identification. Hampering our assessment 
and scholarly efforts in this regard, as noted by Sternberger et al. (2007), "many of the collection methods still 
do not follow best practices in research design nor do the researchers collect data in ways that are consistent 
over time” (p. 79). In the final analysis, then, educators and administrators with an interest in these matters 
must recognize and account for the profound human complexities that interact with a particular international, 
multicultural, or transformative learning experience to produce the outcomes that we ultimately observe. To 
achieve such goals, we must become much more methodologically intentional and sophisticated in our 
assessment research and practice in order to 1) address the overarching questions of who learns what and why, 
and under what circumstances and 2) translate our understanding into interventions that demonstrably 
improve teaching, training, and learning both in and outside of the classroom (e.g., Cultivating the Globally 
Sustainable Self, 2015; Shealy, in press). 
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The Forum BEVI Project 2 

In an attempt to address the above assessment-based complexities, the Forum on Education Abroad 
through its Outcomes Assessment Committee partnered with the International Beliefs and Values Institute 
(IBAVI) in 2007 to design and implement a project using the Beliefs, Events, and Values Inventory (BEVI) in 
order to assess the processes and outcomes of international, multicultural, and transformative learning. There 
were many reasons for doing so. In particular, the Forum on Education Abroad has a strong and 
demonstrable record of supporting research-based activities that have the potential to illuminate relevant data, 
best practices, and appropriate standards for enhancing quality and impact within the field of international 
education (see www.forumea.org/ ). At a complementary level, the mission of the IBAVI is to “explore beliefs 
and values and how they influence actions, policies, and practices around the world” (see www.ibavi.org/) . 
This partnership therefore was ideally suited to the fundamental goals and purposes of both organizations. 
Moreover, given the comprehensive nature of the BEVI and its integrated theoretical and applied emphases, 
this measure was seen as ideally suited to a long-term and large scale research-to-practice agenda. As such, the 
overarching goal throughout this multi-institution, multi-year initiative was to identify findings, applications, 
and recommendations that were of the broadest and deepest relevance to the larger field. From the beginning, 
then, this project was designed deliberately to speak to these larger issues of assessment research and practice 
vis-a-vis international, multicultural, and transformative teaching, training, and learning including, but not 
limited to, the BEVI. 

Toward such means and ends, the Forum BEVI Project participants committed - from the outset - to 
the following principles and practices of assessment research and practice (e.g., Braskamp, Blair, Shealy, & 
Hammer, 2012; Braskamp & Engbert, 2014; Deardorff, 2006; Gerber, Gottlieb, Pysarchik, & Shealy, 2011; 
Pysarchik, Shealy, & Whalen, 2007a, 2007b; Shealy, 2014; Sternberger et al„ 2007): 

1. International, intercultural, and transformative learning is highly complex but measurable. 

2. Any credible assessment project requires sufficient understanding of relevant theory, measures, 
design, data, and analysis by all project members. 

3. The differences and similarities between assessment practice (what is done in the field) and 
assessment research (how we understand what is happening through measurement and 
analysis) must be reconciled. 

4. Appropriate assessment standards and practices must be understood and followed (e.g., ethical 
conduct, IRB approval). 

5. Relevant expertise must be accessed (e.g., research, psychometric, statistical). 

6. Sufficient institutional support must be secured (e.g., administrative, financial). 

7. Processes of project implementation and execution must be consistent with best practices, and 
monitored throughout. 

A final organizational step involved the creation of the Forum - IBAVI Working Group on BEVI 


2 A project of this scope could not have occurred without the deep and abiding support from multiple individuals, 
institutions, and organizations over many years of work. In particular, we gratefully acknowledge the support and 
engagement of all who participated in various phases of the Forum BEVI Project as well as the indispensable 
commitments throughout by the Forum on Education Abroad and International Beliefs and Values Institute. We also 
are grateful for the statistical work of Wenjuan Ma as well as early contributions from Peter DeMichael and Chad Craun, 
and for the programming expertise of Craig Salee, assisted early on in the project by Todd Broadbent. We also gratefully 
acknowledge the editorial assistance of Pamela Roy, former Research and Scholarship Associate, and Joy Milano, 

Research Intern, in the Office of Study Abroad at Michigan State University, both of whom helped address multiple 
aspects of manuscript preparation. Although our process was highly fluid and intensive, it is our hope that the 
applications and implications documented here illustrate what may be learned and accomplished through such sustained 
and collaborative research of this nature. 
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Implementation in Research, Education, and Practice. This working group was charged with: 1) identifying 
and investigating empirical and theoretical questions that may be pursued via the BEYI, which are relevant to 
international, intercultural, and transformative research, assessment, and practice, broadly defined; 2) 
providing guidance regarding how the BEVI may best be used to promote learning, growth, and development 
in various educational, applied, and practical contexts and settings; and, 3) specifying best practices for usage 
of the BEVI’s individual, group, and organizational report system. 3,4 

Forum BEVI Project Pilot Phase 

As envisioned at the outset, this project would include both pilot 3 4 5 and formal phases of implementation. 
As such, beginning in 2007 and completed in 2009, the pilot phase involved administration of the BEVI to 
nearly 2,000 international learning participants in the U.S. and abroad. Statistical analysis narrowed the 
original number of factors on the BEVI from 40 to 18, and nearly 60 items were eliminated during the 
subsequent review process. Reliabilities for most of these scales (i.e., factors) were above 0.80 (no retained 
scale had a reliability of less than 0.75); no factor was retained without an eigenvalue of at least 2; and no item 
was retained with less than a .30 factor loading. Based upon factor analytic and correlation matrix data, as 
well as a Schmid-Leiman transformation (essentially a factor analysis of the factor analysis) (Schmid & 

Leiman, 1957), the 18 scales of the BEVI long version are organized under a series of headings, which 
correspond with the basic theoretical framework of this measure (discussed below). 

Despite such advances, a number of institutions and organizations still wished for a shorter version of the 
BEVI for a number of reasons. First, within the context of higher education in particular, assessment 
demands already were high and student/faculty time was short. Second, although each scale on the long 
version of the BEVI assessed different constructs, these were interdependent with one another (i.e., by design, 
and consistent with the interconnected nature of beliefs, the “oblique” nature of factor rotation parameters 
allowed items to load on more than one factor) (see www.thebevi.com ). Information gleaned from separate 
Exploratory Factor Analyses (EFAs) during its development did much to illuminate how and why specific 
“beliefs, events, and values” were associated together as they were. However, we long had recognized the need 
to move beyond EFA in order to determine by Confirmative Factor Analysis (CFA) if and how the EFA 
structure of the measure held upon administration to a new and separate sample, and to understand the 
relationship among more parsimonious versions of each scale construct. Thus, from 2011 to 2013, the 


3 We gratefully acknowledge the participation of many faculty, staff, and administrators - and thousands of 
students in the U.S. and internationally - who participated in various stages of this project. Lead participants and their 
accompanying institutions included Kris Acheson, Georgia State University; Tamara Baldwin, University of British 
Columbia; Brad Baltensperger, Michigan Technological University; Cynthia Banks, GlobaLinks Learning Abroad (ret.); 
Brian Brubaker, IES Abroad; Stephen DePaul, Terry College of Business, University of Georgia; Jennifer Engel, Loyola 
University Chicago; RT Good, Shenandoah University; Esther Gottlieb, The Ohio State University; Melanie Harding, 
University of British Columbia; Wenjuan Ma, Michigan State University; Gayle Nelson, Georgia State University; 
Hajime Nishitani, University of Hiroshima; Gaby Peschiera-Carl, University of South Carolina; Dawn Pysarchik, 
Michigan State University; Craig Shealy, James Madison University and International Beliefs and Values Institute; 
Christopher Shirley, Georgetown University; Lee Sternberger, James Madison University; Arnd Wachter, Crossing 
Borders Education; Christa Walck, Michigan Technological University; Felix Wang, James Madison University; and 
Elizabeth Wandschneider, Michigan State University. 

4 Selected material in this article (e.g., regarding the BEVI, learning implications) has been adapted from Shealy 
(2014) and Chapters 2, 3, and 4 in the Making Sense of Beliefs and Values book with Springer Publishing (Shealy, in 
press). Wenjuan Ma, Forum BEVI Project statistician, is responsible for the initial development of the “short version” 
psychometrics. 

5 The following colleges, universities, and study abroad program providers, all members of the Forum on Education 
Abroad, participated in the pilot phase of this project: Brethren Colleges Abroad, Dickinson College, IES Abroad, 
International Studies Abroad, James Madison University, Michigan State University, Saint Mary’s College, Saint Olaf 
College, University of South Carolina, University of Texas, and Wells College. 
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developer of the BEVI (Craig Shealy) coordinated a plan and process with the Working Group to develop a 
“short version” of the BEVI. 

Short Version of the BEVI 

Throughout this process, an overarching goal was to determine if a shorter version of the BEVI, with 
substantially fewer items, could be developed in a manner that didn’t compromise the fundamental integrity 
of the measure (i.e., continued to illuminate how and why “beliefs, events, and values” were interrelated as 
they seemed to be via multiple analyses over the years). As any psychometrician will attest, this process was 
painstaking and intensive. Following a data-scrubbing process (e.g., ensuring that duplicated or incomplete 
cases were removed from the database), we identified a sample of 2,331 cases to be used in conducting the 
CFA. This initial phase was more do-able than the next, which required multiple steps to determine which 
items could be eliminated without sacrificing the integrity of the measure. First, we confirmed which items 
were loading on which specific scale. Because a number of the long version scales were measuring higher 
order constructs, it was necessary once again to identify smaller subsets of items (i.e., sub-factors) that 
comprised the larger construct. We used Cronbach's alpha to select the items that could be removed safely 
without significantly impacting the consistency of a particular factor (i.e., scale) or its sub-factors (i.e., sub¬ 
scales). We then used analytic methods aligned with Item Response Theory (IRT) to identify the relative 
contribution of each item to each scale. Again, the overarching goal of this step was to ensure that the short 
BEVI extracted information about respondents that was similar to the information extracted on the long 
BEVI. 6 Although the analyses for this process were relatively straightforward, the challenge lay in the sheer 
volume of data as well as the need to examine all possible permutations among all items and all scales. 
Ultimately, we automated these procedures via a “python program,” which would stop and output results 
whenever the Cronbach's alpha coefficient is equal to 0.7 and allow us to compare the respective shapes of the 
information curve for each version of the BEVI. In other words, to preserve the integrity of the overall BEVI, 
items loading on the short version needed to evidence a similar capacity to identify the same types of 
respondents as did the longer version (e.g., regardless of whether someone strongly agreed or strongly 
disagreed to the items on a particular scale, the short BEVI needed to be able to identify such individuals with 
equivalent sensitivity to the longer version). The end result of this process was the selection of candidate 
items for retention and deletion in the development of the short BEVI. 

But despite this fundamental step forward, we weren’t done yet. That is because the python program 
often “spat out” different multiple item combinations of various short BEVI scales. To figure out which 
combination was best for each scale, we used structural equation modeling (SEM) to test all of these possible 
combinations based upon theoretical propositions that had emerged over years of research with the measure in 
order to explain “what” specific BEVI scales were assessing and why. This process also was highly iterative, 
involving a great deal of back-and-forth dialogue between theoretical and statistical perspectives on this 
measure. Ultimately, we were able to settle on final solutions for all scales that had good statistical and sound 
theoretical properties. At the conclusion of this process, 40 demographic and background variables (as 
opposed to 65), 185 items (as opposed to 336), and 17 scales (as opposed to 18) were retained in the short 
BEVI. 7 Table 1 summarizes core information regarding these scales. 


6 Since most institutions, organizations, and settings are now using the “short version” of the BEVI, and because of 
the extensive analytic process that resulted in this version, it seems likely as of this writing that the “short version” will 
become the primary version of the BEVI. 

7 It should be noted that we retained one scale - Identity Diffusion - that did not meet this .70 threshold (it had an 
alpha of .61). Also, a few items progressing through the first steps did not survive SEM, but were retained nonetheless. 
On both counts, our reasons for doing so were to identify specific combinations of items for the short version of the 
BEVI that had the best reliability while also retaining as much fidelity as possible to the longer BEVI. Along these lines, 
one scale - Global Engagement - appeared to be a sub-factor of a newly named factor, called Meaning Quest, which 
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Table 1 

BEVI Scale Summaries 



Mean 

Std. Deviation 

Cronbach 

Alpha 

Number of 
Items 

Negative Life Events 

2.889 

0.610 

0.862 

9 

Needs Closure 

2.646 

0.290 

0.712 

25 

Needs Fulfillment 

1.892 

0.342 

0.882 

24 

Identity Diffusion 

2.791 

0.322 

0.610 

13 

Basic Openness 

2.108 

0.417 

0.809 

12 

Basic Determinism 

2.887 

0.355 

0.755 

16 

Ecological Resonance 

2.248 

0.524 

0.760 

6 

Self Certitude 

2.122 

0.357 

0.761 

13 

Religious Traditionalism 

2.705 

0.782 

0.903 

5 

Emotional Attunement 

2.175 

0.421 

0.814 

13 

Physical Resonance 

2.200 

0.429 

0.719 

7 

Self Awareness 

1.855 

0.358 

0.810 

12 

Socioemotional Convergence 

1.908 

0.286 

0.877 

36 

Sociocultural Openness 

2.058 

0.287 

0.798 

26 

Global Resonance 

1.719 

0.469 

0.828 

6 

Gender Traditionalism 

2.275 

0.472 

0.828 

11 

Meaning Quest 

1.873 

0.317 

0.831 

19 


As noted above, to explore scale structure, we used SEM to test the relationships between the items and 
constructs, a process that was highly iterative. Table 2 summarizes the final model fit information, which 
indicates that these scales have a relatively good model fit and fit the underlying theory adequately well. 


explains why the short version of the BEVI has one less scale than the long version. Extensive review of item 
combinations also resulted in the renaming of several scales in order to better represent the apparent meaning of each 
factor. Finally, another round of subject matter expert review of the demographic/background items was conducted as 
well, which resulted in the elimination of 25 of these items. 
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Table 2 

Model Fit Information for BEVI Scales 



Chi-square 

df 

p-value 

CFI 

RMSEA 

Negative Life Events 

428.612 

27 

0.000 

0.977 

0.080 

Needs Closure 

2993.316 

225 

0.000 

0.911 

0.073 

Needs Fulfillment 

2855.248 

248 

0.000 

0.912 

0.067 

Identity Diffusion 

28.973 

2 

0.000 

0.983 

0.076 

Basic Openness 

619.225 

54 

0.000 

0.956 

0.067 

Basic Determinism 

536.465 

41 

0.000 

0.927 

0.072 

Ecological Resonance 

456.526 

9 

0.000 

0.967 

0.147 

Self Certitude 

634.634 

62 

0.000 

0.937 

0.064 

Religious Traditionalism 

166.821 

9 

0.000 

0.995 

0.087 

Emotional Attunement 

654.891 

62 

0.000 

0.960 

0.064 

Physical Resonance 

40.557 

2 

0.000 

0.984 

0.091 

Self Awareness 

598.360 

54 

0.000 

0.948 

0.066 

Socioemotional Convergence 

3523.339 

369 

0.000 

0.901 

0.061 

Sociocultural Openness 

2596.628 

225 

0.000 

0.935 

0.067 

Global Resonance 

93.898 

14 

0.000 

0.994 

0.050 

Gender T raditionalism 

765.686 

44 

0.000 

0.948 

0.084 

Meaning Quest 

836.661 

61 

0.000 

0.925 

0.074 


In addition to these key aspects of test development described above, other major activity of the Working 
Group was to present findings of our work over the six years of this project, which included over 20 
publications (e.g., articles, chapters, dissertations), 50 presentations (e.g., symposia, papers, posters), and 
hundreds of separate analyses. The bulk of this article provides a summary of these applications and 
implications. Before such an overview, however, it may be helpful to include additional explication regarding 
the theoretical underpinnings, applications, and scale structure of the BEVI. 


Equilintegration Theory and the Equilintegration Self 

Before describing the BEVI in greater detail (e.g., what it actually measures), a brief overview of its 
theoretical underpinnings - Equilintegration (El) Theory and the El Self - is needed. Essentially, 
Equilintegration (El) Theory seeks to explain “the processes by which beliefs, values, and ‘worldviews’ are 
acquired and maintained, why their alteration is typically resisted, and how and under what circumstances 
their modification occurs" (Shealy, 2004, p. 1075). Derivative of El Theory, and informed by scholarship in 
a range of key areas (e.g., “needs-based” research and theory; developmental psychopathology; social 
cognition; psychotherapy processes and outcomes; affect regulation; theories and models of “self”), the El Self 
seeks to illustrate how the interaction between our core needs (e.g., for attachment, affiliation) and formative 
variables (e.g., caregiver, culture) results in beliefs and values about self, others, and the world at large that we 
all internalize over the course of development and across the life span (see Shealy, in press). 


The Beliefs, Events, and Values Inventory (BEVI) 

Although we’ve described various psychometric and development aspects of the BEVI, we haven’t yet 
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discussed what the BEVI really measures. Concomitant with El Theory and the El Self, the BEVI is a 
comprehensive analytic tool that examines how and why we come to see ourselves, others, and the larger 
world as we do (i.e., how life experiences, culture, and context affect our beliefs, values, and worldview) as 
well as the influence of such processes on multiple aspects of human functioning (i.e., learning processes, 
relationships, personal growth, the pursuit of life goals). The BEVI assesses processes such as: basic openness; 
the tendency to (or not to) stereotype in particular ways; self- and emotional awareness; preferred strategies for 
making sense of why “other” people and cultures “do what they do”; global engagement (i.e., receptivity to 
different cultures, religions, and social practices); and, worldview shift (i.e., to what degree do beliefs and 
values change as a result of specific experiences). BEVI results are translated into reports at the individual, 
group, and organizational levels that include a range of scores, indexes, and narratives. Individuals, groups, 
institutions, and organizations use the BEVI and its report system in a wide range of contexts (e.g., 
educational, mental health, organizational), and for a variety of applied and research purposes (e.g., to track 
and examine changes in worldviews over time; to evaluate and improve educational programs or experiences; 
to cultivate growth and awareness in individuals, couples, and groups; to develop leadership capacity; to 
enhance organizational functioning; to evaluate whether assessment-based goals are met). 

Both the long (336 item) and short (185 item) versions of the BEVI consist of four interrelated 
components: 1) a comprehensive set of demographic/background items that may be modified for particular 
projects; 2) a life history questionnaire, which is built into the measure; 3) two validity and 18 (long version) 
or 17 (short version) scales; and, 4) three qualitative “experiential reflection” items. As a Web-based and 
Likert-type inventory with four response options (Strongly Agree, Agree, Disagree, Strongly Disagree), the 
long version of the BEVI typically requires between 35 and 45 minutes to complete; the short version 
typically requires between 25 and 30 minutes to complete. Items are balanced deliberately to minimize social 
desirability and other response-set confounds (see Geisinger, 2013; Robinson, Shaver, & Wrightsman, 1991, 
1999). Most importantly, the BEVI is not designed to appraise the debatable correctness or incorrectness of 
individual responses, but rather to investigate how overall response patterns predict various processes and 
outcomes, from the beliefs and values associated with ethnocentrism, religious tolerance, partisanship, and 
gender-based practices and policies to issues of openness, self-access, and emotional attunement. 

As noted via multiple chapters in Making Sense of Beliefs and Values - of which this article for Frontiers is 
included as a chapter - the BEVI has been used for a wide range of purposes (see 

http://www.springerpub.com/making-sense-of-beliefs-and-values.html ). Research-focused chapters explain 
how the El model and BEVI method increase our conceptual sophistication and methodological capacity 
across a range of areas: Culture, Development, Environment, Gender, Politics, and Religion. Practice- 
oriented chapters demonstrate how the BEVI is used in the real world across a range of applied domains: 
Assessment, Education, Forensics, Leadership, and Psychotherapy. These implications and applications of the 
BEVI are illuminated by a number of statistical analyses (e.g., ANOVA [analysis of variance], regression, 
structural equation models) as well as the BEVI report system. Evidence of validity is indicated by numerous 
studies demonstrating that the BEVI is able to predict group membership across a wide range of variables 
including demographics such as gender, ethnic background, parental income, political orientation, and 
religious orientation as well as other relevant variables and processes as documented in other chapters of this 
book (e.g., Atwood, Chkhaidze, Shealy, Staton, & Sternberger, 2014; Brearly, Shealy, Staton, & Sternberger, 
2012; Hill, Johnson, Brinton, Staton, Sternberger, & Shealy, 2013; Isley, Shealy, Crandall, Sivo, & Reifsteck, 
1999; Hayes, Shealy, Sivo, & Weinstein, 1999; Patel, Shealy, & De Michele, 2007; Pysarchik, Shealy, & 
Whalen, 2007; Shealy, 2000a, 2000b, 2004, 2005, 2006, 2012; Tabit, Cozen, Dyjak, Pendleton, Shealy, 
Sternberger, & Staton, 2011). For example, El theory, the El Self, and the BEVI offer an integrative 
framework and method for predicting and explaining characteristics that are linked to other global issues such 
as environmental concerns (Patel, 2008) and intercultural awareness (Reisweber, 2008; see also Shealy, 

Bhuyan, & Sternberger, 2012). 
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Although both “long” and “short” versions of the BEVI are available, many users prefer the short version 
mainly due to time savings, as described previously. Therefore, the following overview of BEVI scales is based 
upon the “short version,” which overlaps substantially with the “long version.” Again, the short BEVI consists 
of two validity and seventeen process scales. Although there are many ways to present scale scores, they often 
are presented as a series of colored bars along with a number within each bar. This number corresponds to the 
percentile score between 1-100 that an individual or a group scored on that scale based upon their overall 
response to the items that statistically load on each BEVI scale (i.e., “loading” refers to which items have been 
shown statistically to cluster together on a specific “construct” or scale of the BEVI, and therefore comprise 
the items on that scale). The resulting scores are standardized based upon the means and standard deviations 
for each scale (see Table 1). For group reports, the score is called “aggregate” because it represents the average 
score for all individuals in the group on a specific scale. BEVI results are presented under nine different 
domains as indicated below. A description of each of the BEVI scales, under their respective domains, also is 
provided along with sample items for each scale (in parentheses). 8 In reviewing the following information, 
remember that by design, items may load statistically on a given scale in either a negative or positive direction. 
Thus, if sample items seem to be the opposite of one another, that is both expected and appropriate in terms 
of the psychometrics of inventories such as the BEVI (i.e., positively and negatively loading items both may 
comprise a given scale). 

I. Validity Scales 

• Consistency: the degree to which responses are consistent for differently worded items that are 
assessing similar or identical content (e.g., “People change all the time.” “People don’t really 
change.”). 

• Congruency: the degree to which response patterns correspond to that which would be predicted 
statistically (e.g., “I have real needs for warmth and affection.” “I take my own feelings very 
seriously.”). 

II. Formative Variables 

• Demographic/Background Items: gender, educational level, ethnicity, political/religious 
orientation, income, etc. (e.g., “What is your gender?” “What is your ethnic background?”). 

• Negative Life Events (Scale 1): difficult childhood; parents were troubled; life conflict/struggles; 
many regrets (e.g., “I have had a lot of conflict with one or more members of my family.” “My family 
had a lot of problems with money.”). 

III. Fulfillment of Core Needs 

• Needs Closure (Scale 2): unhappy upbringing/life history; conflictual/disturbed family dynamics; 
stereotypical thinking/odd explanations for why events happen as they do or why things are as they 
are (e.g., “I had a wonderful childhood.” “Some numbers are more lucky than others.”). 

• Needs Fulfillment (Scale 3): open to experiences, needs, and feelings; deep care/sensitivity for self, 
others, and the larger world (e.g., “We should spend more money on early education programs for 
children.” “I like to think about who I am.”). 

• Identity Diffusion (Scale 4): indicates painful crisis of identity; fatalistic regarding negatives of 
marital/family life; feels “bad” about self and prospects (e.g., “I have gone through a painful identity 
crisis.” “Even though we expect them to be, men are not really built to be faithful in marriage.”). 

IV. Tolerance of Disequilibrium 

• Basic Openness (Scale 5): open and honest about the experience of basic thoughts, feelings, and 
needs (e.g., “I don’t always feel good about who I am.” “I have felt lonely in my life.”). 

• Self Certitude (Scale 6): strong sense of will; impatient with excuses for difficulties; emphasizes 
positive thinking; disinclined toward deep analysis (e.g., “You can overcome almost any problem if 


8 The BEVI is a copyrighted instrument. BEVI items, item content, scales, or reports may not be modified, copied, 
disseminated, or published, in whole or part, without the written and express permission of Craig N. Shealy, Ph.D. 
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you just try harder.” “If you play by the rules, you get along fine. 

V. Critical Thinking 

• Basic Determinism (Scale 7): prefers simple explanations for differences/behavior; believes people 
don’t change/strong will survive; troubled life history (e.g., “AIDS may well be a sign of God’s 
anger.” “It’s only natural that the strong will survive.”). 

• Socioemotional Convergence (Scale 8): open, aware of self/other, larger world; thoughtful, 
pragmatic, determined; sees world in shades of gray, such as the need for self-reliance while caring for 
vulnerable others (e.g., “We should do more to help those who are less fortunate.” “Too many people 
don’t meet their responsibilities.”). 

VI. Self Access 

• Physical Resonance (Scale 9): receptive to corporeal needs/feelings; experientially inclined; 
appreciates the impact of human nature/evolution (e.g., “I am a free spirit.” My body is very sensitive 
to what I feel.”). 

• Emotional Attunement (Scale 10); emotional, sensitive, social, needy, affiliative; values the 
expression of affect; close family connections (e.g., “I don’t mind displays of emotion.” “Weakness 
can be a virtue.”). 

• Self Awareness (Scale 11): introspective; accepts complexity of self; cares for human 
experience/condition; tolerates difficult thoughts/feelings (e.g., “I am always trying to understand 
myself better.” “I have problems that I need to work on.”). 

• Meaning Quest (Scale 12): searching for meaning; seeks balance in life; resilient/persistent; highly 
feeling; concerned for less fortunate (e.g., “I think a lot about the meaning of life.” “I want to find a 
better sense of balance in my life.”). 

VII. Other Access 

• Religious Traditionalism (Scale 13): highly religious; sees self/behavior/events as mediated by 
God/spiritual forces; one way to the “afterlife” (e.g., “Without religion there can be no peace.” 

“There is one way to heaven.”). 

• Gender Traditionalism (Scale 14): men and women are built to be a certain way; prefers 
traditional/simple views of gender and gender roles (e.g., “Women are more emotional than men.” 

“A man’s role is to be strong.”). 

• Sociocultural Openness (Scale 15): progressive/open regarding a wide range of actions, policies, 
and practices in the areas of culture, economics, education, environment, gender/global relations, 
politics (e.g., “We should try to understand cultures that are different from our own.” “There is too 
big a gap between the rich and poor in our country.”). 

VIII. Global Access 

• Ecological Resonance (Scale 16): deeply invested in environmental/sustainability issues; concerned 
about the fate of the earth/natural world (e.g., “I worry about our environment.” “We should protect 
the land no matter who owns it.”). 

• Global Resonance (Scale 17): invested in learning about/encountering different individuals, 
groups, languages, cultures; seeks global engagement (e.g., “It is important to be well informed about 
world events.” “I am comfortable around groups of people who are very different from me.”). 

IX. Experiential Reflection Items 

The BEVI is a “mixed methods” measure in that both quantitative (i.e., scales) and qualitative (i.e., free 
response) items are asked during administration and used for purposes of interpretation (e.g., Coates, Hanson, 
Samuel, Ashe, & Cozen, in press; Cozen, Hanson, Poston, Jones, &Tabit, in press). The following three 
qualitative Experiential Reflection Items are included in the BEVI, and completed in written format at the 
conclusion of administration: 

First, please describe which aspect of this experience has had the greatest impact upon you and why? 
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Second, is there some aspect of your own “self’ or “identity” (e.g., gender, ethnicity, sexual 
orientation, religious or political background, etc.) that has become especially clear or relevant to 
you or others as a result of this experience? 

Third, what have you learned and how are you different as a result of this experience? 


The Forum BEVI Project: Applications 

Throughout the Forum BEVI Project, a variety of research questions, assessment projects, and applied 
interventions were examined depending upon the emphases and needs of participating institutions and 
organizations as well as the overarching goals of the project to understand better the processes and outcomes 
of international, multicultural, and transformative learning. Overall, these activities fell under one or more of 
the following eight areas of inquiry and practice: 

1. evaluating learning experiences (e.g., study abroad, multicultural courses, training programs, 
service learning, etc.); 

2. understanding learning processes (e.g., who learns what and why, and under what 
circumstances); 

3. promoting learning objectives (e.g., increased awareness of self, others, and the larger world); 

4. enhancing teaching and program quality (e.g., which experiences, courses, programs have 
what impact, and why); 

5. facilitating growth and development (e.g., of individuals, groups, and organizations); 

6. conducting research (e.g., how, why, and under what circumstances people become more 
“open” to different cultures); 

7. addressing organizational needs (e.g., staff/leadership development); and, 

8. complying with assessment and accreditation requirements (e.g., linking objectives to 
outcomes). 

In the pursuit of such activities, over 500 different analyses were conducted with various subsets of 
participants, both within and across participating institutions, which included the generation of thousands of 
individual reports and dozens of group/organizational reports. Although over 8,000 participants completed 
the BEVI from 2007 - 2013, many of the results below were developed either at the institutional level or on 
the basis of a large, cross-institutional dataset (N = 2331) collected toward the conclusion of the project, 
during 2011 - 2012. 9 Participants primarily included undergraduate students (96.7 percent) and were 
recruited through a range of learning experiences (e.g., study abroad, residential learning communities, 
general education courses with a focus on transformative/multicultural learning). A small portion of graduate 
students (3.3 percent) also was included. The sample typically ranged between the ages of 17 - 26, with an 
average age of 19. However, 3.9 percent fell into the age range of 26 - 62, with another 9 percent falling into 
the age range of 12 - 17. Although the majority of participants reported as U.S. citizens (93.3 percent), non- 
U.S. citizens also were included in the sample (N = 156 or 6.7 percent). Also, participants were drawn from 
38 different countries of origin. Of the sample, 79.1 percent reported as Caucasian with 20.9 percent as non- 
Caucasian (6.6 percent Black or African American; .9 percent American Indian or Alaskan Native; 7.4 percent 
Asian or Pacific Islander; 2.9 percent Hispanic/Latino; 3 percent Other). Finally, from the standpoint of 
gender, 40.8 percent of the sample was female, with 59.2 percent male. All participants were required to 
provide informed consent as determined by multiple Institutional Review Boards processes, and participation 
was entirely voluntary. Participants were not required to complete the BEVI, and could elect to discontinue 


9 Note that some of the scale names differ from the short and long versions of the BEVI. In addition to the 
clarification provided above, please see www. thebevi. com for additional information. 
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participation at any time. Analyses were conducted via SPSS and MPLUS, and consisted of ANOVAs, 
regression analyses, and structural equation modeling (SEM). 


The Forum BEVI Project: Implications for International, Multicultural, and 
Transformative Learning 

Given the scope of this work, it is challenging to select only a small sample of findings from hundreds in 
order to identify the most salient “lessons learned.” Ultimately, it proved most useful to distill all that we 
discovered through this project into a series of 15 implications for international, multicultural, and 
transformative learning: 

1. Who we are affects whether, what, and how we learn. 

2. Education, broadly defined, is associated with - and likely causes - change in beliefs and values. 

3. To understand how effective our interventions are, we must assess who learners are. 

4. It is possible to predict who will be interested in - and satisfied by - a learning experience even 
before the learning experience actually occurs. 

5. Specific analyses may be conducted for purposes of outreach and engagement. 

6. All educators and educational experiences are not equal. 

7. Educational experiences are associated with desirable as well as undesirable processes and 
outcomes across a wide range of constructs that must be measured to be known. 

8. Examining only overall (e.g. aggregate) findings or privileging only a few BEVI scales constitute 
tempting but counter-productive approaches to comprehensive assessment. 

9. Although distinct characteristics of various cross-cultural groups are identifiable, broad-based 
assessment suggests within as well as between group variation. 

10. As Kurt Lewin observed, there is nothing so practical as a good theory. 

11. It is possible to identify the profile or signature of an institution or organization. 

12. Group reports help course instructors, program directors, and administrators understand better 
the nature of their particular class, cohort, or staff. 

13. By jointly utilizing individual and group reports, multiple opportunities for self-assessment, 
enhanced understanding, and group development may be facilitated. 

14. Changes in beliefs and values about self, others, and the world at large appear determined 
largely by the 7Ds (duration, difference, depth, determine, design, deliver, debrief). 

15. Best practice in assessment requires best practice in research. 

We discuss each of these implications below, along with relevant findings, applications, and 
recommendations. 10 

Implication 1. Who we are affects whether, what, and how we learn. Formative variables such as life 
experiences and demographics interact powerfully with the experiences we provide to learners (e.g., study abroad, 
multicultural coursework, growth/development workshops, education in general) to influence the learning processes 
and outcomes that we observe. 


10 The presentation of results in the following section of this article is informed by the March, 2012 Special Section 
of the journal Psychotherapy, entitled “Ethical Issues in Clinical Writing,” Volume 49, Issue 1, pp. 1 - 25 as well as 
H1PAA regulations, APA ethical guidelines, and other best practices for reporting such information. In this regard, 
although all assessment and report results are consistent with original patterns and profiles, key information may have 
been modified on occasion (i.e., specific scores) in order to ensure the anonymity of respondents and/or consistency 
across the long and short versions of the BEVI. 
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N=1016, Chi-square=9141.328, df=1257, p=0.000, RMSEA=0.052, CFI=0.938 


Figure 1. Sociocultural Openness as a mediator between demographic and experiential variables and interest in 
global learning experiences (international, multicultural, and transformative). 

Consider Figure 1, which is a structural equation model (SEM) regarding the relationship between 
Formative Variables, Mediators, and Outcomes. Essentially, SEM is a sophisticated statistical approach that is 
derivative of various methods for establishing predictive pathways among constructs that are thought to have a 
relationship with one another. In other words, SEM allows for theory testing, since this approach evaluates 
the statistical relationship that is theorized to exist between constructs that have achieved a measure of 
reliability and validity. Thus the “model” of structural equation modeling essentially is the theory presumed 
to underlie the relationship among a set of constructs (e.g., Kline, 2010). From the standpoint of the El 
model and BEVI method, the SEM pictured in Figure 1 basically contends that formative variables (e.g., 
antecedent variables that influence why we come to experience self, others, and the larger world as we do) are 
associated with/predictive of mediating variables (e.g., latent variables that help explain underlying 
psychological processes that contribute to various outcomes that we observe), which further predict outcomes 
(e.g., empirically observable outcomes, such as behaviors or verbalizations regarding one’s intention or reality). 

So, from this perspective - and grounded in the El theoretical framework - one’s stated “interest” in 
international or multicultural learning is, in no small part, determined by previous life experiences, which 
shape our “beliefs and values” regarding self, others, and the larger world. This proposed theoretical model 
regarding the predictive linkage among formative variables, mediators, and outcomes, receives strong support 
from Figure 1. Flere, we used Mplus with means and variance adjusted weighted least squares (WLSMV) as 
the estimator. Antecedents in the model are: Negative Life Events; ethnicity measured as a dummy variable 
(0=minority, l=white/Caucasian); family income as an ordinal measure of the annual family income even if 
the participant did not receive financial support from them (l=less than $10,000/year to 10= $175,000 
+/year); and, father’s and mother's education levels (each range from 0=some high school or less to 8=doctoral 
degree). The dependent variable is the level of interest in global, multicultural or transformative learning, 
which is measured by a single question (1= extremely low interest to 7=extremely high interest). 

Results show variability in the impact of formative variables on students’ interest in international and 
multicultural learning. Table 3 below provides the parameters of these linkages. Specifically, Negative Life 
Events and ethnicity are negatively associated with Sociocultural Openness, which means that students 
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experiencing fewer Negative Life Events were more Socioculturally Open, and that Caucasian students are less 
Socioculturally Open than minority students. Moreover, mother's education is positively associated with 
Sociocultural Openness, while family income and father's education are not significantly related to 
Sociocultural Openness. Sociocultural Openness also has a positive influence on students’ interest in 
Table 3 

Factors Impacting Interest in Global Learning: Structural Coefficients, SEM 



Unstandardized 

coefficient 

Standardized 

coefficient 

Negative Life Events ► Sociocultural Openness 

-0.166 

-0.173 

Ethnicity-► Sociocultural Openness 

-0.454 

-0.220 

Family Income-► Sociocultural Openness 

-0.007 

0.025 

Father's Education-► Sociocultural Openness 

0.016 

0.044 

Mother's Education-► Sociocultural Openness 

0.061 

0.156 

Sociocultural Openness ► Global Learning Interest 

1.139 

0.498 


engaging in international, multicultural, and transformative learning experiences. In short, the experience of 
particular formative variables (positive life events, being non-Caucasian, and having a mother with a higher 
degree of education) is predictive of the degree to which one is “open” to the beliefs, values, and practices of 
other cultures, which is further and strongly predictive of one’s interest in engaging with and learning from 
cultures that are different from one’s own. 

Implication 2. Education, broadly defined, is associated with — and likely causes — change in beliefs 
and values. Abundant quantitative and qualitative evidencefrom different types of learning experiences assessed by 
the Forum BEVI Project (e.g, study abroad, multicultural coursework, general education) indicates that students 
are likely to see and experience themselves, others, and the larger world differently following such experiences. 

Contemplate, first, the degree to which “education” influences the relative sophistication that students 
demonstrate in terms of their attributions about self, others, and the larger world, a framework that may 
generally be grouped - from the standpoint of the BEVI - under the domain of “Critical Thinking.” What is 
meant by this construct? Consider a joint statement presented at the 8th Annual International Conference on 
Critical Thinking and Education Reform (Scriven & Paul, 1987), in which critical thinking is defined as “the 
intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, 
and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, 
or communication, as a guide to belief and action” (see https://www.criticalthinking.org/pages/defining- 
critical-thinking/766 ). 11 According to this definition, critical thinking involves a gestalt of competencies that 
helps an individual assess information (in terms of thoughts, feelings, and behaviors) within oneself and one’s 
surroundings, understand and reason with complexity, and utilize this resulting awareness to understand self, 
others and the world in more sophisticated and accurate terms. 

Along related lines, Cottrell (2005) regards critical thinking as a meta-thinking skill or cognitive process 
based on intellectual traits, exposure, educational subjectivity, and the disposition to use such capacities. 
Another comprehensive definition of critical thinking that emphasizes the “self” is offered by Paul and Elder 
(2006), which contends: 

Critical thinking is, in short, self-directed, self-disciplined, self-monitored, and self-corrective 
thinking. It presupposes assent to rigorous standards of excellence and mindful command of their 
use. It entails effective communication and problem solving abilities and a commitment to 


11 Aspects of this section are excerpted and/or adapted from Iyer (2013). 
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overcome our native egocentrism and sociocentrism (p. 4). 

Finally, highlighting the interrelated complexity of this construct, Manaloa, Kusumib, Koyasub, Michitac, and 
Waseda (2013) observe that scholars have defined the construct of “critical thinking” in any number of ways, 
to include: 

• “skilled and active interpretation and evaluation of observations and communications, 
information and argumentation,” (Fisher & Scriven, 1997, p. 21) 

• “correct assessment of statements,” (Ennis, 1962, p. 81) 

• “identification and evaluation of implicit theoretical assumptions,” (Yanchar, Slife, & Warne, 

2008, p. 265) 

• “an attitude or disposition to recognize when a skill is needed and the willingness to apply it” 
(Halpern, 1998, p. 452). 

The importance of critical thinking, and the relative effectiveness of higher education in facilitating its 
development and expression, was brought home dramatically by Academically Adrift (Arum & Roksa, 2011), 
which concluded that student advances in “critical thinking” were modest at best, mainly because of a lack of 
rigor (e.g., limited reading and studying requirements). We appreciate this important contribution to a 
significant question regarding the efficacy of higher education, noting that such issues often come down to 
matters of measurement (e.g., from a quantitative standpoint, which items load statistically on a specific scale) 
and operationalization (e.g., the steps or operations that are assembled to define and demonstrate the 
construct in question, including “critical thinking”). 

In this regard, from the standpoint of the BEVI and its underlying El theoretical framework, the 
construct of “critical thinking” is resonant with several of the above perspectives (e.g., Manaloa et ah, 2013; 
Scriven & Paul, 1987). Specifically, “critical thinking” is evidenced mainly by the degree of attributional 
sophistication regarding why people do what they do as well as why various events or phenomena occur as they 
do, and under what circumstances (Shealy, 2004; Shealy, in press). From a construct level of analysis, on the 
long version of the BEVI, such processes are operationalized via four scales: Causal Closure, Basic 
Determinism, Divergent Determinism, and Socioreligious Closure. Among other applications, through the 
report system described below, the BEVI assists students, instructors, and program directors alike in reflecting 
on these aspects of critical thinking in order to assess and practice greater self-reflexivity toward self, others, 
and the larger world (e.g., Ethics of International Engagement and Service-Learning Project, 2011; Iyer, 

2013). From the standpoint of the Forum BEVI Project, we wished to examine whether the relatively 
straightforward moderator of “number of years in college” was associated with critical thinking as defined by 
the BEVI. 

Structural equation modeling (SEM) again was chosen to analyze the relationship between the number 
of years in college and dimensions of critical thinking. As Figure 2 illustrates, three of the four BEVI scales 
that are intended to measure “critical thinking” are in fact statistically associated with years in college. What 
do such results suggest? Essentially, individuals who score highly on Basic Determinism tend to prefer 
basic/simple explanations for why people are as they are or do what they do (e.g., are inclined towards simple 
explanations of complex, biopsychosocially-mediated phenomena such as ethnic or gender-based differences 
among people). Individuals who score highly on Divergent Determinism tend to eschew commonly accepted 
answers or understandings of phenomena, prefer non-convergent explanations and non-traditional policy 
solutions, are unconventional, tend to question authority, may reflexively be contrarian (i.e., for the sake of 
being so), and resist being pinned down (e.g., will tend to contest the validity or truth of whatever claims are 
being made). Individuals who score highly on Socioreligious Closure tend to have strong religious beliefs and 
great faith, believe that God alone provides happiness/health, and perceive little personal control. In short - 
and mindful of the possibility for correlation/causation confusion - the more years in college, the higher the 
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overall degree of critical thinking, as evidenced by lower scores on Basic Determinism, Divergent 
Determinism, and Socioreligious Closure. 


Years in College 

Key Findings : 

A higher number of 
years impacts 
critical thinking as 
defined by the 
BEVI. 
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Chi-square=1579.701 df=149, p=0.000, RMSEA=0.094, CFI=0.912 
Figure 2. The relationship of “critical thinking” scales on the BEVI to number of years spent in higher education. 

Consider a more fine-grained analysis by examining correlation matrix findings regarding Divergent 
Determinism, the scale that is most strongly negatively associated with number of years of college (i.e., lower 
Divergent Determinism is associated with more years of college). A primary usage of correlation matrices is to 
understand the underlying relational patterns among constructs (i.e., scales) which may relate to one another. 
In so doing, the deliberately oblique factor structure of BEVI scales should be recognized. In other words, as 
is predicted by the El theoretical model, and in keeping with the “web-based” nature of “beliefs and values,” 
from a factor rotation standpoint, items are permitted to load on one or more scales, which in fact is key to 
“making sense” of why beliefs and values are related to each other, and to life events, in the way that they are 
(i.e., as they naturally occur for real human beings) (Shealy, in press). In this regard, we examine in Table 4 
the strongest correlations between Divergent Determinism and other BEVI scales (those at .40 or above). 


Table 4 

Correlation Matrix Findings above .40 between Divergent Determinism and other BEVI Scales 


Scales 

Correlation 

Socioemotional Convergence 

-0.81 

Needs Closure 

0.65 

Emotional Attunement 

-0.58 

Negative Life Events 

0.53 

Sociocultural Openness 

-0.50 

Basic Closedness 

0.45 

Ecological Resonance 

-0.43 

Identity Diffusion 

0.42 
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What do such findings suggest? As Table 4 illustrates, correlation matrix data suggest individuals who 
score highly on Divergent Determinism also: 

• are likely to have greater difficulty “holding complexity,” tending to see the world in “black and 
white” terms (Socioemotional Convergence); 

• are more likely to indicate that core needs (e.g., for attachment, affection) were not met in a 
“good enough” manner (Needs Closure); 

• are less likely to indicate the capacity and inclination to experience affect in self and other, and 
to value its expression (Emotional Attunement); 

• are more likely to report the experience of a high degree of unhappy life experiences, during 
childhood/adolescence or generally in life (Negative Life Events); 

• are less likely to be interested in or open to the beliefs, values, and practices of cultures that are 
different from one’s own (Sociocultural Openness); 

• are more likely to deny or ignore fundamental thoughts, feelings, or needs that generally are 
experienced as “normative” regarding human existence or functioning (Basic Closedness); 

• are less likely to be concerned about or invested in matters that have to do with the environment 
and natural world (Ecological Resonance); and, 

• are more likely to express feelings of confusion and entrapment regarding their current existence 
and future prospects (Identity Diffusion). 


Taken together, these SEM (Figure 2) and correlation matrix findings (Table 4) suggest that critical 
thinking as defined by the BEVI should be understood not only in terms of the relative attributional 
sophistication of human beings (e.g., the ability to understand the complexities of why we think, feel, and do 
what we do or why events happen as they do in the larger world). At a more complex level, it also appears 
that such capacities and inclinations may in fact be associated with our life histories, since a higher degree of 
Negative Life Events and Needs Closure appears to be associated with a lower degree of critical thinking as 
defined by the BEVI, an outcome that would in fact be predicted by the basic needs-based postulates of the El 
model (Shealy, in press; Shealy et al., 2012). In any case, the most basic finding here is that the relative 
sophistication and complexity of attributional tendencies about why people do what they do, or why events 
happen in the world as they do, increases with the more years of college or university that people experience. 
Although subject to further inquiry, and ever mindful of the possibility for correlation/causation confusion 
(e.g., the role of maturational effects), such results suggest that repeated exposure to complex concepts 
combined with multiple experiences in which one is required to understand, reflect upon, and justify the 
rationale for beliefs and values in self, others, and the larger world may in fact be associated with the 
acquisition of more sophisticated attributional frameworks over time. 

Implication 3. To understand how effective our interventions are, we must assess who learners are. 

Attributing positive ratings of learning experiences only or largely to the experience itself, without accounting for who 
students are prior to their engagement in the experience, is neither good practice nor empirically indicated. 

To what degree does exposure to international, multicultural, and transformative learning actually 
change fundamental aspects of an individual’s personality or identity, indeed the very “self” of the learner? 

Put in other terms, does exposure to such learning experiences cultivate the capacity to care in a deeper way 
for self, others, and the larger world (Shealy et al., 2012)? Such questions are complicated in their own right, 
and rendered more so by the fact that individuals who self-select to engage in such experiences may already be 
“in the choir to which we sing” (i.e., they “get” what we are seeking to facilitate even before the learning 
experience begins). Multiple findings from the Forum BEVI Project suggest that “who we are” prior to the 
experience in which we engage may interact profoundly with the experience itself to influence outcomes. 
Consider the following correlation matrix data from one of the “primary” factors on the BEVI, Sociocultural 
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Openness. 

Table 5 

Correlation Matrix Findings above .40 between Sociocultural Openness and other BEVI Scales 


Scales 

Correlation 

Needs Closure 

-0.90 

Ecological Resonance 

0.88 

Socioemotional Convergence 

0.82 

Basic Closedness 

-0.81 

Identity Diffusion 

-0.71 

Emotional Attunement 

0.77 

Socioreligious Traditionalism 

-0.62 

Hard Structure 

-0.58 

Negative Life Events 

-0.57 

Divergent Determinism 

-0.50 


What do such findings suggest? Essentially, individuals scoring high on Sociocultural Openness tend to 
endorse a worldview that may be characterized as liberal, progressive, accepting, culturally attuned, open, 
concerned, and globally oriented. As Table 5 illustrates, individuals who score highly on Sociocultural 
Openness: 

• are more likely to indicate that core needs (e.g., for attachment, affection) were met in a “good 
enough” manner (Needs Closure); 

• are more likely to be concerned about or invested in matters that have to do with the 
environment and natural world (Ecological Resonance); 

• appear more able to experience the world in complex “shades of gray,” rather than in black and 
white terms (Socioemotional Convergence); 

• are less likely to deny or ignore fundamental thoughts, feelings, or needs that generally are 
experienced as “normative” regarding human existence or functioning (Basic Closedness); 

• are less likely to express feelings of confusion and entrapment regarding their current existence 
and future prospects (Identity Diffusion); 

• are more likely to indicate the capacity and inclination to experience affect in self and other, and 
to value its expression (Emotional Attunement); 

• are less likely to endorse a traditional worldview regarding the nature and purpose of religion 
and its centrality to one’s own life (Socioreligious Traditionalism); 

• are less likely to express very strong certitude regarding the correctness of one’s own way of 
seeing self, others, and the larger world, while denying doubts or weakness (Hard Structure); 

• are less likely to report the experience of a high degree of unhappy life experiences during 
childhood/adolescence or generally in life (Negative Life Events); and, 

• are less likely to adopt a reflexively contrarian posture regarding the nature of “truth” or “reality” 
(Divergent Determinism). 


In many ways, a high degree of Sociocultural Openness may be seen as “optimal” in terms of the goals 
that were articulated by various institutions and organizations with which we worked throughout the Forum 
BEVI Project. Again, however, it should be emphasized that a high - or low - degree of Sociocultural 
Openness may serve as cause and/or effect. For example, individuals high in Sociocultural Openness may be 
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predisposed toward study abroad or service learning even before the experience occurs, which therefore may 
function as more of a mediating effect on observed outcomes than whatever experience they encounter. Thus, 
it is very important to assess such attributes before and after a learning experience in order to be able to “tease 
out” the degree to which the experience itself - or who someone already was before the experience - is 
responsible for whatever changes we observe. In any case, multiple findings suggest that the construct (i.e., 
scale) of Sociocultural Openness is in fact associated with the sorts of outcomes that appear to be at the core 
of most international, multicultural, and transformative learning interventions (e.g., study abroad, courses, 
workshops), in that a higher degree of Sociocultural Openness is predictive of a range of real world outcomes, 
such as those described next. 

Implication 4. It is possible to predict who will be interested in — and satisfied by — a learning 
experience even before the learning experience actually occurs. If formative variables (e.g., life history, 
demographics) and mediators (e.g, affective capacity, attributional styles) are assessed with sufficient breadth and 
depth, it is possible to predict statistically who is, and is not, likely to be inclined toward participation in a learning 
experience — and who is, and is not, likely to report that they actually benefittedfrom and/or were transformed by a 
learning experience — even before the experience occurs. 

Such information matters to institutions and organizations for many reasons, including: 1) 
understanding the “profile” of who was and was not inclined to engage in such experiences; 2) providing 
outreach to students who historically were not inclined to participate in such experiences in order to try and 
facilitate their engagement, and; 3) identifying which experiences were more or less likely to elicit interest 
and/or satisfaction. Overall, multiple analyses conducted through different methodologies indicate that we 
were able to ascertain various profiles - both from background information and BEVI scales - regarding who 
was and was not likely to be interested in participating in and/or satisfied by such experiences, even before 
such individuals completed their first course. Consider, for example, Table 6, which identifies a set of 
variables from the BEVI that predict the degree to which respondents will report an interest in specific 
international or multicultural learning experiences. 12 


12 The dependent variable is a student’s level of interest in international education or study abroad. The question is 
as follows: "On a scale of 1-7, with 1 being ‘extremely low’ and 7 being ‘extremely high,’ please indicate your level of 
personal interest in international education or study abroad experiences." The independent variables are demographic 
and experiential variables. Several of the independent variables are dummy variables: gender (0 = male, 1 = female); 
“parents paying for international experience” (0 = parents do not pay, 1 = parents pay); “university provides orientation 
for international experience” (0 = university does not provide, 1 = university provides); “plan to travel abroad” (0 = no 
plan to travel abroad, 1= plan to travel abroad); “plan to take an internationally focused course” (0 = no plan to take a 
course, 1 = plan to take a course); and, “speak a foreign language other than English” (0 = does not speak a foreign 
language, 1 = speaks a foreign language). Another independent variable, the "highest academic degree intended to 
achieve” is coded from 1 = associate degree, 2 = bachelor’s, 3 = master's, 4 = specialist (e.g., Ed.S), 5 = professional (e.g., 
law), 6 = doctoral degree. Also, participants are asked about the number of foreign countries they have previously visited 
(e.g., respondents indicate the actual number of countries they have visited). Background variables include “mother’s 
education” and “family income.” “Mother's education” indicates the highest academic degree of a respondent's mother, 
which ranges from 0 = some high school or less to 8 = doctoral degree. "Family income” is an ordinal variable that 
reflects the average annual income of a student's parents/guardians regardless if the student receives financial support 
from them. Income ranges from 1 to 10, 1 = < $10,000 to 10 = > $175,000. 
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Table 6 

Predictors of Interest in Global Learning Experiences 1 : Multiple Regression Analysis 



Unstandardized 

Coefficients 

B 

Std. Error 

Standardized 

Coefficients 

Beta 

t 

Sig 

(Constant) 

2.864 

0.173 


16.585 

0.000 

Gender (0=M, 1=F) 

Parents are paying for int'l experience 

0.862 

0.094 

0.240 

9.194 

0.000 

(0= Parents do not pay, 1= Parents pay) 
University provides orientation for international 
experience 

0.915 

0.106 

0.238 

8.599 

0.000 

(0=No orientation, 1=Orientation) 

Family income 

0.430 

0.105 

0.110 

4.110 

0.000 

(1=<$10,000 to 10=>$175,000) 

Plan to travel abroad 

-0.073 

0.019 

-0.103 

-3.841 

0.000 

(0=No plan, 1=Has plan) 

Highest academic degree intend to obtain 

0.598 

0.092 

0.165 

6.504 

0.000 

(1=Assoc. degree to 6=doctoral degree) 

Plan to take an int'l focused course 

0.114 

0.030 

0.094 

3.780 

0.000 

(0=No plan, 1=Has plan) 

Number of foreign countries visited 

0.659 

0.102 

0.167 

6.492 

0.000 

(actual number) 

Speak a foreign language other than English 

0.031 

0.014 

0.056 

2.178 

0.030 

(0=Does not speak, 1=Speaks) 

Mother's education 

0.336 

0.088 

0.095 

3.813 

0.000 

(0=Some high school to 8=doctoral degree) 
F 

R 2 

Adi. R 2 

0.074 

71.319*** 

0.415 

0.409 

0.025 

0.080 

3.030 

0.003 


Global learning experiences = international, multicultural, and transformative learning experiences. 


Multiple regression analysis revealed that among the demographic and experiential background factors, 
gender and parents paying for international experiences prove to be the best estimators of students’ global 
interests, based upon the respective standardized coefficients. Specifically, female students are more interested 
in global learning experiences than males. Intuitively logical, those students whose parents paid for their 
international experiences also have a greater interest in global experiences. Similarly, students who plan to 
travel abroad, plan to take an internationally focused course, and speak a foreign language have a greater 
interest in global learning experiences than those who do not. "Highest academic degree intend to obtain" 
and “Number of foreign countries visited” also are positively associated with interest in global learning 
experiences. "Mother's education" is a relatively weak but positive estimator. Finally, “Family income” holds 
a negative association with a student’s interest in global learning, meaning the higher one’s income, the lower 
one’s interest in global learning. The results of this study suggest that a range of BEVI variables can 
statistically predict interest in global learning, even before such learning occurs (i.e., at the planning stage), a 
finding with implications for recruitment, orientation, program development, and institutional planning (e.g., 
if our goal is to “reach” those students who are least likely to study abroad, such data offer perspective 
regarding who such students are most likely and least likely to be, although specific variability within a given 
institution or organization obviously should be considered). 

In many ways, “interest” in international, multicultural, and transformative learning experiences 
represents one bookend of a process that is of great relevance not only to the research community, but to 
program directors, course instructors, and administrators who want to understand whether specific cohorts of 
students are inclined toward, or against, various learning experiences. But the other bookend of considerable 
relevance concerns whether students actually are satisfied by the experiences in which they engaged, and 
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whether the BEVI is able to predict such satisfaction even before the learning experience occurs. As Table 7 
illustrates, a range of formative variables are statistically associated with such satisfaction. Here, the dependent 
variable is a student’s level of satisfaction across multiple international, multicultural, or transformative 
learning experiences. Specifically, on a scale of 1-7 (with 1 being “extremely low” and 7 being “extremely 
high”), the question is as follows: “If you have participated, or are currently participating, in international, 
learning, service, study abroad, travel, or work experiences 13 please indicate your overall level of satisfaction.” 
The independent variables are demographic and experiential variables. 


13 For clarification, several of the variables in Table 7 are not part of the “satisfaction” question on the BEVI, but 
still emerged as relevant for this analysis, albeit moderately (year in university) or weakly (family income, Democrat 
political orientation, gender, and whether parents traveled abroad). “Satisfaction” options on the BEVI regarding 
international, multicultural, or transformative learning include the following: 1) completing an internship with a non- 
El.S. company (in the United States or abroad); 2) living with a roommate from another country or different ethnic 
background; 3) participation in a course with an international / multicultural focus; 4) participation in on-campus 
international/multicultural events; 5) service learning in the United States; 6) service learning abroad; 7) study abroad; 8) 
studying a language other than English; 9) travel to five or more states in the United States; 10) travel abroad; 11) work 
abroad; 12) living in a residence that is international/multicultural and/or language-focused; 13) participating in off- 
campus international or multicultural events; 14) have not previously participated in any of these experiences; and 15) 
other global or international/multicultural experience(s) (please specify). 
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Table 7 


Satisfaction with International Education / Study Abroad Based upon Demographic Variables (Strongest, Moderate, 
and No Association) 
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Multiple regression analysis revealed that among the demographic and experiential background factors, 
the following were most strongly associated with participant satisfaction: a longer length of time participating 
in one or more of the experiences; previous travel to five or more states in the U.S.; previous travel abroad; 
and, currently studying abroad. In addition, the above noted variable of “interest” in international 
experiences - previously living in an international residence hall; and, a greater number of years in higher 
education - all were among the variables associated with a “moderate” degree of “satisfaction.” In short, the 
results of this study suggest that a range of formative variables statistically can predict satisfaction in 
international, multicultural, or transformative learning - broadly defined - even before such learning occurs. 14 
Although the ability to predict “interest” is one matter, the fact that we also can predict satisfaction on the 
basis of formative variables may seem surprising. However, such findings are highly congruent with an El 
perspective, in that abundant evidence indicates that the kinds of life experiences we do and do not have are 
associated inevitably with how we experience self, others, and the world at large, including but not limited to, 
the satisfaction we ultimately report from engaging in a range of learning experiences. 

Implication 5■ Specific analyses may be conducted for purposes of outreach and engagement. Because 
u>e are able to identify subgroups of students who are more or less likely to be inclined toward specific learning 
experiences, it is possible to ojfer targeted opportunities or programs for them that are congruent with institutional 
goals (e.g, to boost study abroad by minority students). Knowing what sorts of learning experiences are most 
impacfulfor which sorts of students also helps clarify and refine interventions that are designed to retain and engage 
students. 

One pressing question both within the field of international education - and in higher education 
generally as well as other performance-based organizational contexts - is whether and how assessment results 
actually matter in the pursuit of various institutional/organizational goals and objectives. From promoting 
individual learning at the student or staff level, to enhancing the quality of courses or programs, to evaluating 
the degree to which institutions or organizations meet their objectives, these “rubber meets the road” issues are 
not only reasonable, but inextricably linked to the demonstration that what we do as educators and “change 
agents” actually matters in the real world (e.g., Blair, Latz, Sternberger, & Shealy, 2014; Braskamp & Engberg, 
2014). Ultimately, of course, administrators are, or should be, concerned with the justifications for our 
actions or policies. Such considerations concern matters of learning impact, of course, but also should have 
implications for matters of quality assurance and cost. For example, presuming the validity of our assessment 
models and methods, and following best-practice efforts to enhance their quality (e.g., Geisinger, 2013; 
Robinson, Shaver, & Wrightsman, 1991), it does not seem unreasonable to use data for the purpose of 
evaluating programs and courses, and indeed, those who direct or teach them. Throughout this project, we 
have seen enormous variance in the quality of such administration and teaching vis-a-vis program and course 
outcomes (see Implication 6 below). Of course, the key is translating the complexity of assessment data into a 
form that fairly and appropriately may be used to make pragmatic decisions. Given the importance of 
“effective educational practices” identified by the Association of American Colleges and Universities 
(AAC&U, 2007), there can be no substitute for valid assessment of such “high impact” offerings to determine 
how they may be improved and whether they are achieving their potential (see Iyer, 2013; Kuh, 2008). 

For example, consider the below finding. In this case, Shenandoah University wanted to ascertain 
whether there are differences between students who did and those who did not apply for a short-term learning 
abroad program (N=57). Such information was of relevance because this institution was interested in 


14 As indicated by other findings in this article, it should be noted that in addition to these formative variables on 
the BEVI, multiple scales also predict “interest” in and “satisfaction” with international, multicultural, and transformative 
learning experiences. For example, in one analysis, Sociocultural Openness, Needs Fulfilllment, Socioemotional 
Convergence, Global Engagement, Ecological Resonance, Causal Closure, Basic Determinism, Self Awareness, and 
Gender Traditionalism all were strongly or moderately predictive of “interest” in such experiences (e.g., Shealy, 2014; in 
press). 
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knowing why a program that was deliberately designed to be accessible to students (e.g., provided by the 
institution at no cost to the student participant) - while also serving as a gateway to other international 
experiences - was not completed by everyone. From the standpoint of the BEVI, one hypothesis is that 
students who applied were more psychologically predisposed to do so. We used a binary variable (students did 
or did not apply to participate) to group the participants into two categories and compared the differences 
between them. As a dependent variable, we ran a Logit Model to detect differences between the two groups. 
What did we find? Compared to non-applicants, program applicants were more open to and interested in the 
attributes and practices of cultures that were different from their own (Sociocultural Openness, b=7.473, p 
=0.039). Results such as these have implications for outreach and engagement with different types of 
students. That is because it is possible not only to know at the outset of study (i.e., during the first year) 
which students are most and least likely to take advantage of the learning experiences offered to them (i.e., 

“the who”), but also to gain a measure of understanding as to the underlying dispositions that mediate for or 
against engagement (i.e., “the why”). In terms of “best practices” for approaching students who are less 
inclined to engage in these types of learning experiences, it is important to appreciate that these psychological 
processes help us to differentiate among students while also offering guidance regarding the kinds of 
interventions that are more likely to “reach” them. Again, in many cases, we sing to the choir, not recognizing 
that the students who enroll in various learning experiences are self-selecting to do so. More challenging is 
reaching out to those students who may not be predisposed initially to such experiences - that is, they are low 
on Sociocultural Openness - but might be willing to move in that direction if we respond to them in an 
informed, sensitive, and sophisticated manner that better accounts for who they are at the outset (i.e., thus 
approaching them where they are rather than where we wish them to be) (e.g., Tabit, Legault, Ma, & Wan, in 
press). 

Along similar lines, another project partner, Dickinson College, wanted to know if there were differences 
on the BEVI between students who declared majors in different areas of study. If so, such differences also 
could have implications for different approaches to engagement and outreach, and help clarify the reasons 
why students were inclined toward different academic trajectories in the first place. In this analysis of 173 
students, interest in global learning experiences was measured by the following question: "On a scale of 1-7, 
with 1 being ‘extremely low’ and 7 being ‘extremely high,’ please indicate your level of personal interest in 
international education or study abroad experiences." Students’ academic majors were divided into four 
categories: social science, arts and humanities, “hard science,” and undecided. A one-way ANOVA was 
conducted to determine if differences existed between students’ interest in global learning experiences across 
multiple academic majors. As Table 8 indicates, no significant differences emerged, a finding that was 
somewhat unique among project participants. Thus, we examined the data more closely. Analysis of the 
means for each major indicates that all students at Dickinson College have a very strong interest in global 
experiences, regardless of academic major. Given the very strong historical and current commitment to global 
education expressed in Dickinson’s mission statement, brand identity, and curriculum, such findings make 
good sense (i.e., at an institution like Dickinson, “interest” in international learning is not an especially good 
predictor of differences between students since most everyone coming to the institution does so with these 
interests in mind). In combination with other results, such findings ultimately led to another implication, 
that institutions and organizations have very different “profiles” or “signatures,” which should be assessed in 
order to apprehend the nature and culture of its members (see Implication 11 below). 



Elizabeth Wandschneider, eta/ 


179 


Table 8 

Interest in Global Learning Experiences 1 by Students’ Academic Major: Dickinson College 



Type II Sum 



Mean 



Source 

of Squares 

Mean 

df 

Square 

F 

Sig. 

Corrected Model 

2.865a 


3 

0.955 

0.688 

0.561 

Intercept 

6605.555 


1 

6605.56 

4758.89 

0.000 

Major 

2.865 


3 

0.955 

0.688 

0.561 

Social sciences 


6.385 





Arts and humanities 


6.313 





Sciences 


6.143 





Undecided or other 


6.082 





Error 

234.58 


169 

1.388 



Total 

6843 


173 




Corrected Total 

237.445 


172 





R 2 = .012 (Adjusted R 2 = -.005) 

Global Learning Experiences = international , multicultural , and transformative learning experiences. 

Notes. 1. Post hoc tests indicate there are no significant differences among the majors. 

2. Items are measured on a 4-point scale (1=strongly disagree - 4=strongly agree) 

Implication 6. All educators and educational experiences are not equal. The fact that all learning 
interventions and interveners are “not equal” should promote personal, professional, and institutional development 
and engagement, as well as a renewed commitment to improvement, rather than discouragement, defensiveness, or 
denial. 

Although intuitively reasonable, and certainly a focus of emphasis within the academy, it would seem 
that there is a general consensus that those who teach, direct, or lead various learning experiences differ in 
their relative effectiveness. For example, the fact that we expend considerable energy evaluating the 
effectiveness of teaching faculty suggests an overarching recognition that such matters are integral to student 
success, course and program quality, and the pursuit of institutional and organizational goals. Such matters of 
teacher quality are not just of concern within higher education, but are central to all individuals who assume a 
teaching role at the K-12 level as well (e.g., Cultivating the Globally Sustainable Self, 2015; Ellis, Lee, & 
Wiley 2009; InTASC, 2011; Welch, Pitts, Tenini, Kuenlen, & Wood, 2010). Even so, for a variety of reasons, 
it is difficult to tease apart the relative impact of “teacher” from “learner” much less the interactions between 
the two that serve to mediate and moderate learning. Thus, one intriguing and consistent finding that we 
believe has long-term relevance is the apparent differences that emerged in learner outcomes as a function - at 
least in part - of who was teaching and/or leading a particular learning experience. For example, consider the 
following juxtaposition between two different Time 1 / Time 2 group report profiles by two different 
individuals who were responsible for leading the same learning experience (see Figure 3). Flow do we 
interpret such findings? First, it is important to appreciate that this juxtaposition is not based upon truly 
random assignment to either instructor. Rather, as is often the case with large required courses, these students 
simply were assigned to one or another section on the basis of a number of factors (e.g., their own course 
schedules; adviser decisions and enrollments), which are not controlled for in this convenience sample. Thus, 
any number of variables might have affected the results we observe. However, in this particular analysis, the 
basic process for assignment of students to various sections was not pre-determined (i.e., was more or less 
random in a non-experimental sense, in that there was no specific logic or plan to assignment of students to a 
specific section). More importantly, from our review of Time 1 / Time 2 BEVI reports across different 
institutions and programs or experiences, such results do not appear to be anomalous in that we repeatedly see 
substantial differences in Time 1 /Time 2 reports across instructors or directors who are teaching or leading 
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experiences that ostensibly are designed to be “the same.” That said, the most fundamental point we offer in 
this regard - which is at the core of all we have observed throughout this project - is that we simply cannot 
understand who learns what and why, and under what circumstances, if we do not simultaneously take into 
account and assess multiple variables, including teacher characteristics (e.g., dispositions), student 
characteristics (e.g., life histories), and the quality and nature of the learning experience (e.g., how it is 
taught/delivered, where, and for what purposes) (e.g., Cultivating the Globally Sustainable Self, 2015). So, 
we recommend reviewing the patterns revealed in Figure 3 not so much as definitive, but rather as illustrative 
of the kinds of differences that we have observed, and that warrant further study in the years to come. So, in 
the context of these caveats, let’s focus on a few key points of comparison. Specifically, consider differences in 
Figure 4 on Negative Life Events, Self Awareness, and Global Engagement across these two instructors. 

Essentially, on these three Time 1 /Time 2 comparisons, students of Instructor 1 report, by the 
conclusion of the course, that they had experienced a greater degree of Negative Life Events, are more Self 
Aware, and are more Globally Engaged. The exact opposite Time 1/Time 2 profile is associated with 
Instructor 2 (students report substantially fewer Negative Life Events, much less Self Awareness, and much 
less Global Engagement). Although hard and fast conclusions are not warranted for the reasons noted above, 
since the goals for each section of the course ostensibly were the same (e.g., to produce greater awareness of 
different cultures), these results should give pause at the very least, as they suggest students are reporting 
radically different experiences at the conclusion of the course. 

Why? Among other possible hypotheses, the most parsimonious would seem to be the skill through 
which course content was conveyed and course experiences were facilitated as well as overarching fidelity to 
the course objectives that were pursued. Such matters of content and process are, of course, at the heart of 
teaching and learning. As our annual processes of faculty evaluations across higher education institutions 
indicate, we certainly believe that faculty differ in their instructional competencies, a supposition that would 
seem to receive at least potential support from such findings. At the same time, many other variables could 
account for such differences as well, including fundamental differences in the nature of various cohorts (e.g., 
most faculty who have taught for any duration are likely to experience substantial differences in the make up 
or nature of cohorts from semester to semester). 

And yet, the dramatic difference in directionality from Time 1 to Time 2 across the above three scales - 
and others - suggest that such phenomena should be studied more closely over time, particularly because we 
have observed such differences in outcomes across instructors, programs, and institutions. The results suggest 
not only a deep and largely neglected need to examine differences in the knowledge, skills, and dispositions of 
those who are in an “instructional” realm, broadly defined, but also how such findings may be used to 
promote growth and development of instructors and facilitators of change at multiple levels and in various 
sectors of the educational system. That is to say, if used with care and skill, such findings may help individual 
instructors understand better the characteristics of their group (i.e., through group reports of their students) as 
well as what is happening from the beginning to the conclusion of a learning experience. If skillfully 
facilitated, such findings should help teaching faculty reflect upon and cultivate capacities over the short- and 
long-term that are more likely to be effective in terms of facilitating learning, development, and change (e.g., 
Cultivating the Globally Sustainable Self, 2015). 
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Instructor 1 


Instructor 2 


1. Negative Life Events (T2) | 
2. Needs Closure (Tl) 

2. Needs Closure (T2) 

3. Identity Closure (Tl) 

3. Identity Closure (T2) 

4. Basic Closedness (Tl) 

4. Basic Closedness (T2) 

5. Hard Structure (Tl) | 

5. Hard Structure (T2) | 

6. Causal Closure (Tl) | 
6. Causal Closure (T2) | 

7. Basic Determinism (Tl) ( 
7. Basic Determinism (T2) | 
8. Divergent Determinism (Tl) | 

8. Divergent Determinism (T2) | 

9. Socioreligious Closure (Tl) | 
9. Socioreligious Closure (T2) | 

10. Emotional Attunement (Tl) 

10. Emotional Attunement (T2) 

11. Positive Thinking (Tl) 

11. Positive Thinking (T2) 
12. Self Awareness (Tl) 
12. Self Awareness (T2) 
13. Socioemotional Converg. (Tl) ( 


13. Socioemotional Converg. (T2) | 
14. Sociocultural Openness (Tl) 

14. Sociocultural Openness (T2) 

15. Socioreligious Tradition (Tl) 
15. Socioreligious Tradition (T2) 

16. Gender Traditionalism (Tl) 

16. Gender Traditionalism (T2) 

17. Ecological Resonance (Tl) 
17. Ecological Resonance (T2) 

18. Global Engagement (Tl) 
18. Global Engagement (T2) 


High 


1. Negative Life Events (Tl) 





High 


1. Negative Life Events (Tl) 


1. Negative Life Events (T2) | 15 
2. Needs Closure (Tl) 

2. Needs Closure (T2) 

3. Identity Closure (Tl) 

3. Identity Closure (T2) 

4. Basic Closedness (Tl) 

4. Basic Closedness (T2) | 

5. Hard Structure (Tl) | 

5. Hard Structure (T2) j 1 2 

6. Causal Closure (Tl) 

6. Causal Closure (T2) 

7. Basic Determinism (Tl) 

7. Basic Determinism (T2) 

8. Divergent Determinism (Tl) I 

8. Divergent Determinism (T2) | 

9. Socioreligious Closure (Tl) | 

9. Socioreligious Closure (T2) | 

10. Emotional Attunement (Tl) 

10. Emotional Attunement (T2) 

11. Positive Thinking (Tl) 

11. Positive Thinking (T2) 

12. Self Awareness (Tl) 

12. Self Awareness (T2) | 

13. Socioemotional Converg. (Tl) | 

13. Socioemotional Converg. (T2) | 

14. Sociocultural Openness (Tl) 

14. Sociocultural Openness (T2) 

15. Socioreligious Tradition (Tl) 

15. Socioreligious Tradition (T2) 

16. Gender Traditionahsm (Tl) 

16. Gender Traditionahsm (T2) 





17. Ecological Resonance (Tl) | 
17. Ecological Resonance (T2) H 
18. Global Engagement (Tl) | 
18. Global Engagement (T2) | 


Figure 3. Comparison of Time 1 and Time 2 profiles for two different instructors of the same course. 


1. Negative Life Events (Tl) 
1. Negative Life Events (T2) 

12. Self Awareness (Tl) 
12. Self Awareness (T2) 

18. Global Engagement (Tl) 
18. Global Engagement (T2) 



1. Negative Life Events (Tl) 

32 

1. Negative Life Events (T2) 

15 



12. Self Awareness (Tl) 
12. Self Awareness (T2) 




18. Global Engagement (T1)J 
18. Global Engagement (T2)J 


Figure 4. Comparison of Tl / T2 results for two different instructors on the same course across three BEVI scales. 


Ultimately, ensuring instructor competence, effectiveness, and impact is an ever-relevant issue for 
institutions and organizations. We therefore suspect that using such rich data to analyze the quality of 
instructors, directors, and leaders could in fact result in significant improvements to course or program 
offerings over the long term. Moreover, if students or other learners repeatedly are exposed to instructors or 
directors who consistently achieve results that are below - or above - those of their colleagues, we have a very 
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real opportunity to intervene in a data-driven manner in order to 1) promote the instructional, curricular, or 
programmatic quality of a specific learning experience; 2) encourage ongoing professional development that 
may be tracked over time; 3) recognize and explore “excellence” when it repeatedly occurs in terms of “better” 
learning processes outcomes, to understand what actually is happening in real time between teachers, trainers, 
and learners; and, 4) make changes when all else fails - and after all good faith attempts to improve matters - 
so that learners are not repeatedly subjected to subpar instructors, directors, or leaders who do not appear to 
be facilitating acceptable learning processes and outcomes. 

At the very least, these findings suggest the need to examine such outcome differences in order to 
understand what the interactions may be among teacher competencies, student characteristics, and the nature 
of various learning experiences that are designed to promote “change” (e.g., Cultivating the Globally 
Sustainable Self, 2015). The need to consider such matters becomes especially important when one realizes 
that exposure to international, multicultural, or transformative learning may not always result in “good” 
outcomes at Time 2 (see Implication 7). For example, we have observed Time 1/Time 2 instructional profiles 
which may show a much higher degree of Emotional Attunement and Self Awareness, but lower Sociocultural 
Openness, at Time 2. Contrast that outcome with its mirror opposite of a much lower degree of Emotional 
Attunement and Self Awareness at Time 2, but higher Sociocultural Openness. Which outcome is better? 
One’s answer probably depends upon one’s goals in terms of a course, program, or workshop, but arguably, a 
higher degree of Emotional Attunement and Self Awareness may actually be a function of exposure to content 
and experiences that are highly activating of the “self,” which may not necessarily translate into greater 
openness toward the sociocultural “other” (e.g., Tabit et ah, in press) at the conclusion of a learning 
experience. Over the long term, however, such an outcome may bode better than the opposite outcome, 
which finds our learner less Emotionally Attuned or Self Aware, but reporting more Sociocultural Openness. 
As always, the reasons for such discrepant outcomes should be a point of exploration and contemplation - 
which may be much better interpreted and understood in the context of overall scale scores and qualitative 
findings - rather than a basis for hard and fast conclusions on their own. Along similar lines, and as 
explicated in Implication 7 below, we strongly recommend longitudinal analysis (e.g., Time 3, Time 4 
administration), mainly to hone in on what is happening over time. That is because it may well be that an 
initial decrement on various scales represents the initial “reaction” to an overwhelming experience, which must 
be metabolized over time in order to ascertain longer-term effects. 

Implication 7. Educational experiences are associated with desirable as well as undesirable processes 
and outcomes across a wide range of constructs that must be measured to be known. Depending upon the 
nature and quality of learning, growth, or development experiences — and the life histories, worldviews, and 
capacities of those who deliver and engage in them — measurable processes and outcomes may or may not be in the 
anticipated direction. Moreover, “change” may be occurring in ways that are and are not expected. If such processes 
and outcomes are not measured (e.g., because they aren’t appreciated or seen as relevant, or because they are assumed 
to be happening as a matter of course), we may miss information that could in fact be most important about what 
actually is happening for learners, irrespective of what we think is, or should be, happening. Thus, as educators, 
researchers, and administrators, we should embrace the fundamentally experimental nature of our work. We may 
learn as much from what doesn’t work as what does, but only if we measure a sufficient range of relevant constructs. 

Oftentimes, understandably, the focus within the larger field of higher education, including 
international, multicultural, and transformative learning as a whole, is on “what works.” Such an emphasis 
makes good sense, of course, since it behooves us to develop, implement, and promote learning experiences 
that demonstrably impact our students, institutions, and organizations in a value-added manner. But 
sometimes the most useful findings that emerged from the Forum BEVI Project involved learning experiences 
that didn’t seem to accomplish their intended goals, especially when these unexpected results were discussed 
openly and productively by project partners. To take one example of several, students enrolled in a course 
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Table 9 

Degree of Worldview Shift 




Time 


Gender 



Scale 

Pre 

Post 

F 

Female 

Male 

F 

Causal 

Closure 

1.229 

1.317 

7.780 
(1, 133)** 

1.196 

1.351 

11.595 
(1, 133)*** 

Basic Determinism 

1.718 

1.856 

9.187 
(1, 134)*** 

1.615 

1.958 

19.196 
(1, 134)*** 

Emotional 

Attunement 

3.013 

3.133 

6.920 
(1, 133)** 

3.257 

2.889 

13.581 
(1, 133)*** 

Global 

Engagement 

2.647 

2.579 

4.454 
(1, 134)* 

2.691 

2.535 

7.662 
(1, 134)** 


designed to produce greater sociocultural openness and appreciation for diversity 15 were selected for an 

analysis aimed at better understanding the degree to which individuals do or do not benefit from such a 
learning experience. As described in the syllabus, the purpose of the course was to provide students with the 
necessary tools to understand and appreciate the diverse cultures that they will likely encounter throughout 
their personal and professional lives. The study employed a one-group pretest/posttest design across four 
sections of the course. Students (N = 137) who chose to participate in the research project were provided 
with a username and password and asked to complete the long version (336-item) BEVI pretest at the 
beginning of the semester. At the conclusion of the course, students were again asked to complete the BEVI 
as a posttest measure. Analysis of pre-post data was undertaken across all BEVI scales using repeated measures 
analysis of variance (ANOVA). This study was exploratory in nature, and as such, research questions focused 
mainly on whether Time 1/Time 2 differences could be observed, and if so, on which BEVI scales. Given the 
goals of the course, it was anticipated that some changes would be observed - in a positive or desired direction 
- particularly on scales tapping greater Sociocultural Openness and Global Engagement. 

As indicated in Table 9, results suggest that the course, although designed to facilitate learning outcomes 
such as tolerance for cultures different from one’s own, appears to be associated with the opposite effect. 

More specifically, upon completion of the course, students appeared to be more rigid in their belief systems, 
more inclined to endorse simple causal attributions regarding why human beings do what they do, more 
emotionally aware and activated, and less open to developing a deeper engagement with other cultures than 
they had been at course entry. It may, of course, be mistaken to assume that these real-time/short-term 
correlations will bear out over the long-term. Moreover, it cannot be concluded that the course is “causing” 
these changes or even that greater “negative” changes might have emerged had the course not occurred. All 
that said, however, it is striking that theoretically desirable attributes (e.g., openness, engagement) diminished 
over the duration of the semester since such outcomes are antithetical to the explicit goals of this course. 
Moreover, it should be noted that additional qualitative data (e.g., responses on the three free-response items 
of the BEVI; course evaluations) corroborate these quantitative data, suggesting that at least some aspects of 
this particular course should perhaps be reconfigured to focus, for example, more on learning process and less 
on content delivery, especially at the outset of the course (e.g., see Tabit et al., in press). 

This Time 1/Time 2 within-group analysis provided one of our project partners an opportunity to use 
the BEVI to gauge and compare the relative effectiveness of this required course. Combined with other data, 


15 Adapted from Tabit, Legault, Ma, &Wan (in press) from Shealy, C.N. (Ed.) (in press). Making sense ofbeliefs and 
values. New York, NY: Springer Publishing. 
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profile results may be used to improve this teaching and learning experience over time, modify extant courses, 
develop new approaches, and so on. Moreover, such results provide an opportunity for self-assessment and 
professional development, as group reports may help course instructors, program directors and administrators 
better understand the nature of their particular class or cohort and their own effectiveness as instructors. For 
example, as described below, the report system for the BEVI allows instructors to examine group 
characteristics in detail and to consider how such variables impact the ways in which course content is 
experienced, which offers an opportunity to modify and/or tailor one’s approach to teaching in order to 
proactively respond to the dynamics of the larger group and its subgroups. Along similar lines, by tracking 
experience-based outcomes over time, it is possible to identify those educator/leader dispositions that appear 
to be most and least effective in terms of meeting teaching or program outcomes. Such information may be 
quite helpful in terms of understanding areas of focus for personal and professional development. 

At the same time, as noted above, it also is important to appreciate the longitudinal nature of “change” as 
is suggested by the El model (Shealy, in press). That is to say, “negative” results from a learning experience 
may be due to multiple factors, including but by no means limited to 1) the quality of instruction or 
facilitation, 2) the nature of a particular cohort, and/or 3) the fact that exposure to content and processes that 
are “disequilintegrating” may show up on the BEVI in one way at Time 2, but be completely different at Time 
3. Research currently underway is following cohorts beyond a Time 2 administration in order to examine 
both longitudinal processes that occur over time (at Time 3 and beyond) and cross-sectional differences that 
may characterize cohorts across different years. 

Consider, in this regard, Figure 5 below, which juxtaposes Time 1, Time 2, and Time 3 BEVI Group 
Report profiles for the same group of study abroad students (N = 20). As this figure illustrates, although some 
aspects of theTl/T2 comparison move in a direction that presumably would be desirable at the conclusion of 
a study abroad experience (e.g., Self Certitude drops from the 42 nd to the 28 th percentile; Gender 
Traditionalism drops from the 38 th to the 27 th percentile), other scales move in a direction that presumably 
would not be desirable given the goals of the study abroad experience (e.g., Needs Fulfillment drops from the 
54 th to the 39 th percentile; Self Awareness drops from the 78 th to the 63 rd percentile; and Sociocultural 
Openness drops from the 76 th to the 62 nd percentile). As noted in Figure 5, T1/T2 administrations occurred 
between 3-4 months after initial administration of the BEVI at the beginning of the study abroad 
experience. So, what happens if we let at least half-a-year pass before assessing these same students again? 

Flere we see a highly intriguing flip on a number of key scales. For example, in Figure 5 below, consider three 
scales noted above that presumably went in the “opposite direction” of what reasonably could be anticipated 
(or at least hoped for) at Time 2: first, Scale 3. Needs Fulfillment (which measures openness to experiences, 
needs and feelings; deep care/sensitivity for self, others, and the larger world); second, Scale 11. Self 
Awareness (which measures a tendency toward introspection, acceptance of self-complexity, care for the 
human experience/condition, and tolerance of difficult thoughts/feelings); and third, Scale 15. Sociocultural 
Openness (which measures progressiveness/openness regarding a wide range of actions, policies, and practices 
in the areas of culture, economics, education, environment, gender, global relations, and politics). As 
illustrated below, on all three of these scales (among others that might be reviewed as well from Figure 5), 
results now solidly are in the direction of what might be anticipated (or hoped for) at the conclusion of a 
study abroad experience. Flow do we explain such findings? From an Equilintegration or El perspective, the 
“7Ds” of belief/values transformation (see Implication 14) explicitly acknowledge that multiple factors 
interact simultaneously to determine how, for whom, and under what circumstances “change” occurs (Shealy, 
in press). Moreover, the greater the discrepancy between one’s original “Formative Variables” and those to 
which one is exposed next (e.g., how the individual experiences a culture with which they are deeply familiar 
as compared to how that same individual experiences a different and unfamiliar culture), the greater the 
degree of potential “affective/cognitive shut down” occurs, as the “self” strives essentially to protect “its self” 
from the intensely experienced shock of such exposure, which may in qualitative terms be described as 
“amazing,” but nonetheless exerts a toll; thus, the individual simply may need time and space to consolidate 
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and “make sense of beliefs and values” once again in a way that is “equilintegrated” into one’s newly 
reorganized self. Thus, we see a Time 3 profile that largely is a mirror image of Time 2, and in some ways, a 
poignant reminder of what the human self endures as we all go about the business of living, which is felt that 
much more dramatically when experiencing “high impact” learning like study abroad (e.g., Iyer, 2013; Kuh, 
2008). 

From the standpoint of the BEVI and its report system, the basic point to keep in mind here is that the 
interface between the complexities of being human and the complexities of assessment require us to adopt 
measurement strategies that do not wag the dog; in other words, let us first be who we are, and then and only 
then, seek to capture the meaning of our experience vis-a-vis assessment (e.g., by assessing such evolving 
human phenomenology over time, rather than “just once,” as such an approach is more likely to allow for our 
human complexity to be apprehended as it naturally manifests in the real world) (Shealy, in press). At the 
same time, project results suggest that we cannot assume that “negative” results at Time 2 are due simply to 
the “disequilintegrating” processes that may occur as a result of challenging learning, growth, and 
development experiences; it may well be that the two other variables noted above - relative 
instructional/facilitator quality and/or cohort differences - may in fact account for much of the variance in 
this regard (i.e., the fact that learners may report qualitatively different responses at Time 3 than at Time 2 
should not automatically be cited to deflect a focus from instructor/course quality and/or variability among 
cohorts). In short, our results suggest that if we really want to understand why we observe the measurable 
results that we do over time, our commitment to assessment must occur over the long-term, and 
simultaneously be open to examining a wide range of interacting variables that may well have personal as well 
as programmatic and institutional/organizational implications. 
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T1/T2/T3 Aggregate BEVI Profile from a Sample of Study Abroad Students 

(3-4 Months between T1 and T2 Administrations; 10-22 Months between T1 and T3 Administrations) 


N=20 


1. Negative Life Events 


2. Needs Closure 


3. Needs Fulfillment 


4. Identity Diffusion 


5. Basic Openness 


6. Self Certitude 


7. Basic Determinism 


8. Socioemotional Convergence 


9. Physical Resonance 


10. Emotional Attunement 


11. Self Awareness 


12. Meaning Quest 


13. Religious Traditionalism 


14. Gender Traditionalism 


15. Sociocultural Openness 


16. Ecological Resonance 


17. Global Resonance 
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Figure 5. Comparison of Tl / T2 / T3 BEVI group report results for the same group of study abroad students. 
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Implication 8. Examining only overall (e.g. aggregate) findings or privileging only a few BEVI scales 
constitute tempting but counter-productive approaches to comprehensive assessment. Aggregate findings can be 
misleading because they may mask or wash out “opposing” data for subgroups, which, when compiled, “cancel” each 
other out and become invisible. Thus, it is essential that the reporting system be able to account for change processes 
that may be occurring both within 1) demographic subsets of learners and 2) educators who are leading specific types 
of learning experiences. Likewise, ignoring some of the scales to focus on only a few (i.e.. Sociocultural Openness) 
fails to recognize the rich and complex processes students undergo, the relationships between these processes, and the 
importance of assessing other aspects of the learning process, which may be as — if not more — important in 
explaining why we do, or do not, observe particular findings. 

It was heartening early on in the Forum BEVI Project to see results which clearly illustrated not only that 
change was emerging from Time 1 to Time 2 administrations, but in ways that would be expected. Consider, 
for example, Figure 6 below, a Time 1/Time 2 analysis (N = 19) from Brethren Colleges Abroad (BCA), 
which mirrors the Time 1/Time 3 Profile above from Figure 5, and emphasizes a related finding: it always is 
important to consider one’s unique results in relation to the specific instructor, group, and context in which a 
learning, growth, or development experience occurs. In many ways, this profile of this small N — from 
students who experienced a range of study abroad programs in different locales - represents what might be 
seen as optimal. In addition to the interesting finding that 1) Negative Life Events goes down (from the 33 rd 
to the 25 th percentile), these students would appear 2) to be "clearer" about who they are and where they are 
going in life (Identity Closure decreases from the 45 th to the 30 th percentile); 3) to become more sophisticated 
and nuanced at the level of attributions made about self, others, and the larger world (e.g., Basic Determinism 
decreases from the 87 th to the 56 th percentile); 4) to demonstrate greater introspection and interest in “how 
people work” (Self Awareness increases from the 28 th to the 48 th percentile); 5) to report fewer stereotyped 
notions regarding who males and females are and should be (Gender Traditionalism decreases from the 73 rd to 
the 59 th percentile); and, 6) to evidence a greater degree of interest in encountering different cultures and 
practices around the world (Global Engagement increases from the 15 th to 39 th percentile), from Time 1 to 
Time 2. 



Frontiers: The Interdisciplinary Journal of Study Abroad 


188 


Congruency Score T1 
Congruency Score T2 

1. Negative Life Events (Tl) 

1. Negative Life Events (T2) 

2. Needs Closure (Tl) 

2. Needs Closure (T2) 

3. Identity Closure (Tl) 

3. Identity Closure (T2) 

4. Basic Closedness (Tl) 

4. Basic Closedness (T2) 

5. Hard Structure (Tl) 

5. Hard Structure (T2) 

6. Causal Closure (Tl) 

6. Causal Closure (T2) 

7. Basic Determinism (Tl) 

7. Basic Determinism (T2) 

8. Divergent Determinism (Tl) 

8. Divergent Determinism (T2) 

9. Socioreligious Closure (Tl) 

9. Socioreligious Closure (T2) 

10. Emotional Attunement (Tl) 

10. Emotional Attunement (T2) 

11. Positive Thinking (Tl) 

11. Positive Thinking (T2) 

12. Self Awareness (Tl) 

12. Self Awareness (T2) 

13. Socioemotional Converg. (Tl) 

13. Socioemotional Converg. (T2) 

14. Sociocultural Openness (Tl) 

14. Sociocultural Openness (T2) 

15. Socioreligious Traditional) 

15. Socioreligious Tradition (T2) 

16. Gender Traditionalism (Tl) 

16. Gender Traditionalism (T2) 

17. Ecological Resonance (Tl) 

17. Ecological Resonance (T2) 

18. Global Engagement (Tl) 

18. Global Engagement (T2) 



Figure 6. T1/T2 Group Report (N = 19). 
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Again, although such a profile may in fact be “optimal,” a fundamental problem with such Time 1/Time 
2 assessment analyses, no matter what measures are in use, is that such a report may obscure or wash out 
subgroup differences within the larger group. In other words, aggregate presentations of data do not allow for 
a more fine-grained analysis of what actually is happening within the group, not only at a scale level, but in 
terms of multiple mediators and moderators of change (e.g., background/demographic information) that may 
affect who learns what and why, and under what circumstances. As such, although interesting, a simple Time 
1/Time 2 analysis of the group as a whole for any assessment measure, including the BEVI, would probably 
not be sufficient to understand the complex and interacting within group factors that ultimately influence the 
nature and degree of any changes that were observed. Moreover, imagine a Time 1 /Time 2 profile that is the 
opposite of that pictured in Figure 6. Would we therefore conclude the learning experience was a complete 
failure? That could be the case, but at the very least, it would be helpful to do further statistical analysis to 
determine what exactly is happening within facets of the larger group. It could be, for example, that some 
groups may share similarities due to self-selection processes that might lead subsets of students to seek a study 
abroad or multicultural learning experience whereas others would not. In short, our assumption has been that 
multiple and substantial differences would characterize various subgroups within the larger group. Hundreds 
of analyses have confirmed the validity of this most basic proposition, which has substantive implications for 
the interpretation and application of BEVI Time 1/Time 2 reports, and, we would argue, for the very nature 
of how Time 1 and Time 2 evaluation of learning processes and outcomes occurs, regardless of which 
measure(s) are used. 

Consider the following analysis, from one of our project partners, the University of British Columbia 
(UBC) in Canada. Here, in a series of Group Report juxtapositions (N = 36), we are comparing Aggregate 
Profile results with those of Profile Contrast. By way of orientation, the description of each of these indices is 
as follows: 16 

Profile Contrast illustrates how different and similar the group is across all 17 BEVI scales via the lowest 
30 percent, middle 40 percent, and highest 30 percent of full scale scores. The full scale score is summative of 
scores from the seven domains of the BEVI under which the 17 process scales are clustered: I. Formative 
Variables; II. Fulfillment of Core Needs; III. Tolerance of Disequilibrium; IV. Critical Thinking; V. Self 
Access; VI. Other Access; and, VII. Global Access (Shealy, in press). Profile Contrast is “key” to interpreting 
whether and to what degree groupings by low, medium, and high full scale scores are associated with different 
elevations on specific BEVI scales. For example, such information may be helpful in interpreting the 
variability within a specific group, which may range from minimal to substantial. Profile Contrast also may 
help users comprehend how and why subgroups within the larger group show changes in similar and different 
directions in the context of learning, growth, and development experiences. Thus, this index is considered a 
more robust and nuanced measure of the differential impact of specific learning experiences than are 
aggregated indices, which may obscure subgroup differences or cancel out changes that are in fact occurring 
among subsets of the larger group. 

Aggregate Profile presents aggregated background information and domain scores for the full scale 
scores of the entire group (i.e., 1st - 100th percentile). In other words, this is the profile we are suggesting 
indeed is interesting and relevant (i.e., a version of this type of “aggregate profile” typically is relied upon when 
engaging in group-based Time 1/Time 2 assessment) - but should not be used alone. It is most useful for 
Time 1 assessment (i.e., understanding where the overall group is at the outset of a learning, growth, or 
development experience) and to track group results over time (i.e., from year to year). However, it generally 
should not be used alone to determine if a learning, growth, or development experience is or is not effective 
(e.g., at Time 2 assessment) mainly because it may obscure subgroup differences and/or cancel out changes 


16 The following information for Aggregate Profile and Profile Contrast is excerpted and/or adapted from the 
“Description of Tables and Indices” section of BEVI Group Reports. 
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that actually are occurring within various subsets of the larger group. From the standpoint of the BEVI, the 
best index to determine what is happening in this regard (i.e., between the lowest 30 percent, middle 40 
percent, and highest 30 percent of full scale scorers) is Profile Contrast. 

To illustrate these points, consider the following comparisons of Aggregate Profile and Profile Contrast 
results across five BEVI scales for this sample from UBC: Negative Life Events; Identity Diffusion; 17 Self 
Certitude; Religious Traditionalism, and Sociocultural Openness. To facilitate interpretation, in figures 7-11 
below, we have juxtaposed the three Time 1/Time 2 scale scores for Profile Contrast (lowest 30%, middle 
40%, and highest 30%) with the Aggregate Profile score for each respective scale. In this way, it is possible to 
compare subgroup differences and similarities (Profile Contrast) with the overall score (Aggregate Profile) for 
each scale in this series of five separate analyses. 

Consider Figure 7 below. In the first place, note that Aggregate Profile for Negative Life Events (NLE) 
goes up from the 48 th to the 55 th percentile from Time 1 to Time 2. Flowever, on Profile Contrast, the Lowest 
30 percent and middle 40 percent (70 percent of the overall sample) actually go down from Time 1 to Time 2 
on Negative Life Events. A basic and overarching conclusion from such results is that Profile Contrast is a 
much more robust and nuanced analysis of what is actually happening within a group than is the Aggregate 
Profile. So, we must be very careful in our interpretation of Time 1/Time 2 findings on any measure - 
including but not limited to the BEVI - in order to ensure that we are apprehending what actually is 
happening within subsets of the overall group, as such subgroups may actually be responding very differently 
from one another as a result of exposure to an international, multicultural, or transformative learning 
experience. 

Note also the very interesting finding - observed across multiple Group Report analyses - that the 
perception of one’s life history and background may be differentially affected by one’s point of departure prior 
to engaging in an international, multicultural, or transformative learning experience. This observation is 
highly congruent with a narrative framework that how we experience ourselves, others, and the larger world is 
mediated strongly by the life experiences we have and may not “exist” in absolute or inviolable terms (Shealy, 
in press). In other words, our memory and experience of our “past” are highly dependent upon events that 
occur in the present and future. And yet, this process may unfold differently for individuals depending upon 
their point of departure (i.e., how they are “organized” affectively or cognitively prior to the learning 
experience). For example, as Figure 7 illustrates, individuals who report a greater degree of “Negative Life 
Events” at Time 1 subsequently show a decrease in such a report at Time 2 (three to four months after the 
international/multicultural learning experience) whereas individuals who report a lesser degree of “Negative 
Life Events” at Time 1 subsequently show an increase in such a report at Time 2. What do such findings 
suggest? Although we emphasize that such findings are common, but not universal, these outcomes suggest 
that prolonged exposure to radically different “formative variables” (e.g., a different culture, context, language, 
religion, etc.) interacts with “who we were” at the outset of the experience to produce a sort of “progression to 
the mean” effect. Such findings imply further that those who experienced their past as especially “negative” or 
“positive” aren’t so sure that that was the case following the completion of this intensive learning experience 
(i.e., relative to what they now have experienced via a program, course, workshop, etc., their past may now be 
experienced by them as “better” or “worse” than what they originally believed). 


17 For purposes of clarification, note that some scale names / results are from the “long” version of the BEVI 
whereas others are from the “short” version of the BEVI. For example, the name of this scale is “Identity Closure” on the 
long version, but “Identity Diffusion” on the short version. Additional chapters in Making Sense of Beliefs and Values 
(e.g., chapter 4) provide further clarification regarding processes by which the BEVI and its scales were developed (see 
Shealy, in press; http://www.springerpub.com/making-sense-of-beliefs-and-values.htmD . 
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Lowest Middle Highest Aggregate 


Figure 1. Comparison between lowest, middle, and highest Profile Contrast scores with the Aggregate Profile score 
on the Negative Life Events scale from the BEVI. 



Figure 8. Comparison between lowest, middle, and highest Profile Contrast scores with the Aggregate Profile score 
on the Identify Diffusion scale from the BEVI. 
















































Frontiers: The Interdisciplinary Journal of Study Abroad 


192 


Consider Figure 8. From an interpretive standpoint, the Aggregate Profile and Profile Contrast results 
on Identity Diffusion amplify the above point. Recall that “Identity Diffusion” indicates the degree to which 
individuals report a “painful crisis of identity,” and are “fatalistic regarding negatives of marital/family life,” 
and “feel ‘bad’ about one’s self and prospects.” If we only reviewed the Time 1/Time 2 Aggregate Profile 
(which again, takes the average of all participants), we might conclude that no change occurred on this 
fundamental aspect of how one experiences self from the beginning to the conclusion of the experiences (i.e., 
the Time 1/Time 2 Aggregate Report remains at the 18 th percentile). Such a conclusion would be ill-advised 
as there are clear differences between subgroups on this core construct of the BEVI. Specifically, although 
none of these subgroups are elevated on this scale - an important point to consider vis-a-vis interpretation of 
the group as a whole - the lowest and middle “full scale” cohorts would appear to become substantially clearer 
on who they are and where they are going in life, whereas those who believed they were the clearest on such 
aspects of self at the beginning of the experience report markedly less clarity by the conclusion of this 
experience. The reasons for such findings warrant further exploration, but one possibility is that by dint of 
their exposure to a context and culture that are substantially different to what they were accustomed, each 
subgroup may “balance” what they previously believed and valued against what they saw in terms of the 
realities that others faced. Such a process may differentially be associated with greater clarity for those who 
were relatively unclear about who they were and where they were going versus those who, relatively speaking, 
felt clearer about such matters at the outset, but became less convinced of their clarity by the end of the 
experience. 

Such tentative conclusions receive additional support via the following comparison between the 
Aggregate Profile and Profile Contrast for Self Certitude, which measures the degree to which individuals 
experience a “strong sense of will,” are “impatient with excuses for difficulties,” tend to emphasize “positive 
thinking,” and are “disinclined toward deep analysis” regarding why they or others behave as they do or why 
events happen as they do in the larger world. As the findings of Figure 9 illustrate - and again, the group 
overall demonstrates a low degree of this construct - overall findings (from Aggregate Profile) illustrate that 
individuals report becoming less certain regarding such matters overall following an international, 
multicultural, or transformative learning event. Flowever, very important differences - at least for this group 
- emerge, in that those who showed the highest “full scale” profile (i.e., were the most inclined to be open and 
engaged regarding the experience they were about to have) showed the greatest drop in certitude by the 
conclusion of their experience. The lowest full scale group (those that were least inclined) also showed a drop, 
whereas the middle 40 percent showed no change at all. Overall, it would appear that these students became 
“less certain” overall, although those in the middle appeared not to change on this core measure of certitude 
over the course of their experience. 
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Lowest Middle Highest Aggregate 


Figure 9. Comparison between lowest, middle, and highest Profile Contrast scores with the Aggregate Profile score 
on the Self Certitude scale from the BEVI. 

Importantly, global matters of “certitude” may not apply equally to all belief/value constructs that are 
integral to one’s larger worldview. For example, the following finding also has emerged on several occasions, 
and has to do with specific subsystems of belief. Consider Figure 10, which addresses the degree to which one 
is “highly religious,” tends to see “self/behavior/events as mediated by God/spiritual forces,” and contends that 
there is “one way to the ‘afterlife’.” Interestingly, here, the Aggregate Profile is a good representation of how 
the lowest and highest full scale subgroups respond, but does not reflect results for the middle 40 percent of 
this group. How do we interpret such findings? Although subject to further inquiry, a preliminary hypothesis 
is that individuals who are most invested in a traditional religious worldview experience the greatest challenge 
to the black and white nature of such a framework when exposed to beliefs and values that are markedly 
different from their own. Interestingly, such a finding does not correspond with an increase in “Identity 
Diffusion,” as noted above (quite the contrary), which suggests that such transitions are not experienced as 
cognitively/affectively confusing in terms of why the world works as it does, but perhaps, are clarifying 
regarding who we are and who we are striving to become. 
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Lowest Middle Highest Aggregate 


Figure 10. Comparison between lowest, middle, and highest Profile Contrast scores with the Aggregate Profile score 
on the Religious Traditionalism scale from the BEVI. 

Finally, for those engaged in study abroad, service learning, multicultural education, or other educational 
interventions that are designed to facilitate greater openness to different ways of believing in or behaving 
around the world, the following findings, from Figure 11, are worth noting. Specifically, observe that this 
group of 36 students from the University of British Columbia would appear to be extremely socioculturally 
open at the beginning of their learning experience. Although there is a 10 percentage point difference at Time 
1 between the lowest and highest full scale scores, it should be emphasized that this group really is “the choir,” 
in that those engaged in trying to promote the beliefs, values, and practices of a “global citizenry” have a very 
ready audience. So, looking only at scales on the BEVI which measure such outcomes isn’t the most relevant 
area of focus for this particular cohort. In other words, if our goal is “sociocultural openness,” this group 
already is there at Time 1. 























Elizabeth Wandschneider, eta! 


195 


98 97 97 

96 



Lowest Middle Highest Aggregate 


Figure 11. Comparison between lowest, middle, and highest Profile Contrast scores with the Aggregate Profile score 
on the Sociocultural Openness scale from the BEVI. 

Thus, a fundamental recommendation from the Forum BEVI Project is to look far beyond traditional or 
expected indices of what we are trying to facilitate, and instead delve much more deeply into the complex 
interactions that mediate and moderate why and how we experience self, others, and the larger world as we 
do. That is, as international, multicultural, and transformative educators, scholars, and administrators, we 
need to include, but look beyond “sociocultural” beliefs, values, and practices in order to understand whether, 
how, for whom, and to what degree our learning experiences are impacting learners. A focus only on 
“intercultural” learning would miss the broader implications of the above findings, which illustrate that 
learners are being impacted at multiple levels of self (Shealy, in press). If we do not ask about these aspects of 
how self, others, and the larger world are experienced - to include but not be limited to “intercultural” beliefs 
and values - we miss apprehending the breadth and depth of all that we are impacting via the learning 
experiences we create. For example, if we only looked at Sociocultural Openness, it would be tempting to 
observe simply that, “Well, they were open at the beginning and are pretty much the same at the end.” Such a 
simplistic conclusion ignores the rich changes that are in fact occurring - as the other four above comparisons 
reveal - as well as the complex and meaningful interactions that are taking place with subgroups of the larger 
group. The essential point here is that no specific construct should be privileged above any other, regardless of 
the primary educational goal we are seeking to promote. Just as our learners are not only “cultural learners,” 
so should our assessment instruments measure much more than “cultural learning” alone. 

Along these lines, we have quite often observed the intriguing finding that BEVI participants’ reports of 
their own history become more “negative” over the course of a few weeks or months (between Time 1 and 
Time 2 administrations). This phenomenon may occur even while participants simultaneously report a 
greater degree of Emotional Attunement or Self-Awareness at the conclusion of the experience, which suggests 
two intriguing possibilities that are reminiscent of several implications above. First, consistent with the 
constructivist paradigm, it may well be that our present experiences alter our past perceptions under certain 
circumstances such as study abroad. Second, certain experiences may activate emotional arousal or promote 
self-awareness, but not increase, or even decrease, global engagement. Thus, if we only ask about “cultural” 
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beliefs and values, we may well miss the most important - and perhaps the most interesting and relevant - 
aspects of what actually is happening within our “learners” when they are exposed to different cultures. In 
short, we need to examine multiple “parts” and the overarching “whole” simultaneously in order to apprehend 
the experiential gestalt of learning, both at the individual and group level. In so doing, we are much more 
able to grasp the underlying interactions within and among various cohorts of learners, which will help us 
understand what is happening and why, and inform whatever decisions we make regarding changes or 
developments to pursue (e.g., curricular, staffing, faculty, programmatic, administrative, organizational, 
institutional). 

As a final point under this implication, recall that the BEVI includes three “free response” qualitative 
questions - as a mixed methods measure - which allow respondents to offer their perspectives regarding 
matters of growth and change, or the lack thereof, in their own words. The importance of mixed methods 
approaches is well documented in terms of assessment best practice (e.g., Creswell & Plano-Clark, 2007; 
Hanson, Leighton, Donaldson, Terjesen, & Shealy, in press; Teddlie & Tashakkori, 2009), the legitimacy of 
which has been confirmed to us as well on multiple occasions throughout the Forum BEVI Project. 

Qualitative responses provide a window into values-based similarities and differences among various cohorts 
regarding their initial or concluding expectations or perceptions regarding a learning experience. Such 
information may be very helpful in ensuring that program directors or course instructors are better attuned to 
what students or other participants are thinking and feeling vis-a-vis the learning experience, while using such 
information to facilitate more skillful programmatic or instructional interventions. Qualitative data also may 
provide key insight regarding why quantitative data emerge as they do. For example, when profile scores move 
in directions that are the opposite of what was expected or desired from Time 1 to Time 2 (suggesting that the 
overall experience may have been more “negative” than “positive”), qualitative responses are crucial to 
interpreting what may be happening, particularly if several students/participants report similar experiences. 
Specifically, when responses are relatively “neutral” or open at Time 1 (e.g., “I haven’t had an international 
experience yet”; “I don’t know, but look forward to the class”,) - and become indifferent or negatively toned 
upon Time 2 BEVI administration, at the conclusion of the experience (e.g., “I’ve had enough and can’t wait 
to get home”; “This class was terrible”) - we gather important information that may help explain the 
directionality of scale scores. Finally, along similar lines, qualitative responses both help explain quantitative 
findings and highlight the complexity of international, multicultural, and transformative learning processes 
and outcomes. For example, consider the following three prototypical response excerpts to qualitative 
question 3 on the BEVI: “What have you learned and how are you different as a result of this experience?”: 

• I realize how privileged I have been in my life. 

• I now see why my country is the best place to live in the world. 

• Who I thought I was is very different from what I think now. 

When juxtaposed with quantitative scale scores, such responses not only represent a sort of 
phenomenological shorthand for how a student or participant has experienced a program, course, or 
workshop, but also offer a window into why response profiles may be what they are, particularly if patterns 
emerge across cohorts. For example, a student who concludes that they now know the “U.S. is best” likely 
had a very different experience and/or was a very different person at the outset of the experience than someone 
who now recognizes how “privileged” they are as compared to how they thought or felt at the beginning of the 
experience. Again, such responses illustrate a fundamental finding of this entire project: There are marked 
differences between students, faculty, directors, cohorts, and experiences - at the level of beliefs, values, and 
life experiences - that all interact together to influence the sort of “outcomes” that emerge at the conclusion of 
a learning experience. In short, qualitative data allow us to “make sense” of the complexity of international, 
multicultural, and transformative learning in ways that may be highly complementary to, and illuminating of, 
quantitative data. 
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Implication 9. Although distinct characteristics of various cross-cultural groups are identifiable, broad- 
based assessment suggests within as well as between-group variation. A focus on differences between cultural 
groups is interesting and relevant, but may obscure the fact that greater variability may occur more often within, 
rather than between, different cultural groups. Such a conclusion has important implications for how we 
understand “human nature” and make sense of why we become who we become. 

One major question of relevance to the BEVI Working Group was the overarching issue of whether and 
how this measure could be used in countries and cultures different from the United States. That is because 
the adaptation and usage of “Western” tests and measures in NonWestern contexts presents a range of 
challenges, which center largely on equivalency issues and underlying epistemologies of tests and measures 
developed in the U.S., which may or may not be consonant with those of other countries and cultures. From 
our perspective, there need not be an ineluctable schism between North/South, Western/non-Western 
methods and measures. Instead, our “differences” offer a fascinating and rich opportunity for us all to listen 
and learn from each other, ultimately responding in a collaborative, consensus-based manner that is inclusive, 
pluralistic, and culturally sensitive as well as methodologically and psychometrically defensible via assessment 
research and practice (e.g., Hanson et ah, in press; Harkness, 2003; Moghaddam, Harre, & Lee, 2007; Wild, 
Grove, Martin, Eremenco, McElroy, Verjee-Lorenz, & Erikson, 2005). 

Consistent with such a perspective, five points should be emphasized vis-a-vis cross-cultural usage of the 
BEYI. First, many BEVI items actually were identified outside of the U.S. in the context of clinical, research, 
and educational work with a highly diverse international population (Shealy, 2004). Second, every effort has 
been made throughout the development and evaluation of this measure to be mindful of cross-cultural issues 
of usage and application, and to strive for “equivalency” of usage across cultures and contexts. Multiple 
Subject Matter Experts (SME) have reviewed BEVI items at different points in the development of this 
instrument, including non-English language first/non-U.S. experts in international, multicultural, and 
transformative learning in order to identify possible issues of bias and/or non-relevance across countries and 
cultures. Third, from a theoretical perspective, the BEVI deliberately strives for neutrality regarding the 
“rightness” or “wrongness” of responses to BEVI items, allowing respondents to answer on a four-point Likert- 
type scale. 18 As such, the fact that respondents from a “different” culture or context might tend to respond 
one way or another to various subsets of BEVI items, or even the BEVI as a whole, is precisely what the 
measure is designed to detect (i.e., differences and similarities between various subgroups). In other words, if 
such differences emerged across cultures and contexts, the most parsimonious explanation for such differences 
would be that the BEVI is detecting such belief/value differences rather than being biased toward, or against, 
one group or another. Fourth and similarly, based upon hundreds of analyses thus far, there is reason to 
believe that within group variation on the BEVI will be equivalent or greater than between group variation. 

For example, as the Decile Profiles below illustrate, although some culturally salient patterns may emerge on 
the BEVI, which are indeed relevant and meaningful (i.e., on beliefs/values regarding gender, religion, etc.), 
the general tendency is for broad-based populations (i.e., from a country) to show greater within than 
between group variation on many of the constructs that the BEVI measures. Fifth, in the final analysis, such 
matters ultimately are empirical in nature as we have to determine whether there are systematic differences at a 
population level that inadvertently skew results one way or another. Although further analysis of such issues 
remains ongoing, evidence thus far suggests that when administered and interpreted in an appropriate 
manner, the BEVI indeed appears able to be used across cultures and contexts to identify how various groups 
- even with very different life histories and worldviews - experience self, others, and the larger world. 19 


18 See chapter 4 in Making Sense of Beliefs and Values (Shealy, in press) for a discussion of the design and structure of 
the BEVI, including the rationale for not including an “undecided” or “neutral” option. 

19 Reasons for cultural differences and similarities between groups are considered in several chapters in Making Sense 
of Belief and Values (e.g., see Tabit et al., in press). 
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Consider, for example, the following three Decile Profiles that are drawn from students in three different 
countries/global regions under the auspices of universities that were part of the Forum BEVI Project. 20 If 
there are radical differences across these three cultures, they presumably would manifest in patterns of 
responding across various scales on the BEVI (i.e., would be markedly different and/or skewed in various 
directions). Take a moment to review these three profiles and the dispersion of responses across the deciles for 
various BEVI scales. 


Table 10 

Three BEVI Decile Profiles from Students in Three Different Countries/Regions 
Country / Region 1 


1. Negative Life Events 

17% 

20% 

13% 

\0 

O'' 

OO 

6% 

14% 

3% 

6% 

9% 

4% 

2. Needs Closure 

54% 

15% 

9% 

7% 

3% 

3% 

3% 

3% 

1% 

1% 

3. Needs Fulfillment 

2% 

2% 

3% 

7% 

6% 

11% 

9% 

20% 

19% 

22% 

4. Identity Diffusion 

36% 

18% 

17% 

0% 

11% 

0% 

0% 

6% 

3% 

9% 

5. Basic Openness 

6% 

8% 

6% 

11% 

7% 

11% 

15% 

8% 

18% 

12% 

6. Self Certitude 

12% 

9% 

10% 

14% 

9% 

9% 

9% 

15% 

6% 

7% 

7. Basic Determinism 

20% 

17% 

19% 

9% 

8% 

9% 

7% 

2% 

5% 

4% 

8. Socioemotional Convergence 

1% 

4% 

3% 

5% 

10% 

7% 

11% 

14% 

23% 

23% 

9. Physical Resonance 

0% 

0% 

0% 

1% 

1% 

5% 

3% 

13% 

58% 

20% 

10. Emotional Attunement 

6% 

3% 

7% 

9% 

8% 

17% 

9% 

13% 

18% 

10% 

11. Self Awareness 

1% 

0% 

0% 

1% 

1% 

3% 

18% 

8% 

27% 

42% 

12. Meaning Quest 

4% 

3% 

13% 

8% 

8% 

10% 

8% 

20% 

10% 

14% 

13. Religious Traditionalism 

0% 

25% 

18% 

6% 

18% 

8% 

8% 

8% 

6% 

4% 

14. Gender Traditionalism 

24% 

15% 

18% 

5% 

7% 

9% 

7% 

6% 

3% 

5% 

15. Sociocultural Openness 

2% 

3% 

1% 

3% 

6% 

6% 

9% 

13% 

11% 

46% 

16. Ecological Resonance 

5% 

2% 

6% 

7% 

15% 

15% 

11% 

9% 

8% 

23% 

17. Global Resonance 

2% 

5% 

6% 

8% 

0% 

7% 

7% 

13% 

35% 

17% 

Deciles: 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 


20 The Decile Profile essentially divides up each scale on the BEVI into chunks of ten percent — deciles — ranging 
from the basement to the ceiling of each scale. In so doing, Decile Profile illustrates the percentage of individuals from 
the larger group that fall into each decile, from the lowest (1 - 10%) to the highest (90 - 100%), for each BEVI scale. 
Also, for interpretive purposes, the darker the shade of gray, the greater the percentage of individuals who cluster in a 
particular decile. 
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Country / Region 2 


1. Negative Life Events 

9% 

17% 

4% 

9% 

9% 

17% 

13% 

13% 

9% 

0% 

2. Needs Closure 

30% 

13% 

9% 

9% 

13% 

4% 

13% 

0% 

9% 

0% 

3. Needs Fulfillment 

0% 

0% 

13% 

13% 

4% 

9% 

9% 

30% 

9% 

13% 

4. Identity Diffusion 

17% 

9% 

4% 

0% 

4% 

0% 

0% 

26% 

13% 

26% 

5. Basic Openness 

13% 

30% 

0% 

22% 

9% 

4% 

13% 

0% 

9% 

0% 

6. Self Certitude 

0% 

4% 

0% 

9% 

9% 

17% 

9% 

17% 

0% 

35% 

7. Basic Determinism 

4% 

17% 

22% 

13% 

9% 

9% 

9% 

4% 

0% 

13% 

8. Socioemotional Convergence 

0% 

0% 

17% 

4% 

9% 

17% 

0% 

9% 

26% 

17% 

9. Physical Resonance 

0% 

0% 

0% 

13% 

4% 

30% 

0% 

39% 

13% 

0% 

10. Emotional Attunement 

4% 

4% 

13% 

13% 

26% 

9% 

4% 

13% 

13% 

0% 

11. Self Awareness 

0% 

0% 

0% 

13% 

9% 

13% 

9% 

4% 

30% 

22% 

12. Meaning Quest 

0% 

4% 

17% 

4% 

0% 

17% 

4% 

30% 

0% 

22% 

13. Religious Traditionalism 

0% 

0% 

4% 

4% 

0% 

9% 

17% 

17% 

17% 

o 

sp 

O'' 

14. Gender Traditionalism 

13% 

4% 

13% 

4% 

13% 

30% 

0% 

9% 

0% 

13% 

15. Sociocultural Openness 

0% 

0% 

0% 

4% 

4% 

0% 

0% 

9% 

26% 

57% 

16. Ecological Resonance 

0% 

4% 

9% 

0% 

9% 

13% 

17% 

9% 

17% 

22% 

17. Global Resonance 

0% 

0% 

4% 

4% 

0% 

9% 

17% 

13% 

39% 

13% 

Deciles: 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
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Country / Region 3 


1. Negative Life Events 

4% 

25% 

12% 

12% 

4% 

12% 

4% 

12% 

sO 

0 s 

OO 

4% 

2. Needs Closure 

50% 

8% 

17% 

12% 

8% 

0% 

0% 

0% 

0% 

4% 

3. Needs Fulfillment 

0% 

8% 

4% 

12% 

12% 

4% 

12% 

29% 

8% 

8% 

4. Identity Diffusion 

8% 

29% 

21% 

0% 

21% 

0% 

0% 

12% 

8% 

0% 

5. Basic Openness 

0% 

17% 

4% 

29% 

4% 

4% 

12% 

8% 

8% 

12% 

6. Self Certitude 

4% 

25% 

4% 

17% 

0% 

17% 

4% 

21% 

8% 

0% 

7. Basic Determinism 

21% 

12% 

0% 

21% 

4% 

12% 

12% 

4% 

4% 

8% 

8. Socioemotional Convergence 

4% 

4% 

8% 

0% 

21% 

8% 

21% 

8% 

21% 

4% 

9. Physical Resonance 

0% 

0% 

0% 

4% 

0% 

25% 

8% 

s© 

O'' 

OO 

m 

21% 

4% 

10. Emotional Attimement 

0% 

4% 

21% 

17% 

8% 

8% 

21% 

12% 

4% 

4% 

11. Self Awareness 

0% 

0% 

0% 

17% 

4% 

8% 

12% 

8% 

25% 

25% 

12. Meaning Quest 

12% 

4% 

21% 

12% 

8% 

4% 

8% 

21% 

8% 

0% 

13. Religious Traditionalism 

4% 

33% 

12% 

12% 

17% 

8% 

8% 

4% 

0% 

0% 

14. Gender Traditionalism 

17% 

17% 

21% 

0% 

17% 

12% 

4% 

4% 

8% 

0% 

15. Sociocultural Openness 

0% 

0% 

0% 

4% 

4% 

8% 

17% 

12% 

12% 

42% 

16. Ecological Resonance 

0% 

4% 

8% 

17% 

25% 

8% 

4% 

4% 

12% 

17% 

17. Global Resonance 

0% 

8% 

4% 

0% 

0% 

4% 

8% 

17% 

50% 

8% 

Deciles: 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 


Since were dealing with different countries and regions, it makes sense to look first at Sociocultural 
Openness, which measures a tendency toward “progressive” and “open” beliefs and values regarding “a wide 
range of actions, policies, and practices in the areas of culture, economics, education, environment, 
gender/global relations, politics.” Are there differences across these three groups? Consider the percentage of 
the sample that falls in the tenth decile (e.g., at the 90 th percentile or higher on this scale): Country 1 = 46 
percent; Country 2 = 57 percent; Country 3 = 42 percent. Clearly, participants from all three of these 
countries/regions are extremely high on Sociocultural Openness as measured by the BEVI. How about a 
couple of other variables, which might reflect underlying cultural dynamics/issues, such as Religious 
Traditionalism and Gender Traditionalism? What do we see at this level? Essentially, it would appear that 
Country/Region 2 is the most “conservative” or “traditional” regarding matters of religion and gender. 
Specifically, 30 percent of Country/Region 2 participants fall in the 90 th percentile on Religious 
Traditionalism compared to four percent for Country/Region 1 and zero percent for Country/Region 3. And, 
there is much wider dispersion on Gender Traditionalism for Country/Region 2 than either 1 or 3, which 
generally show low levels of Gender Traditionalism. Does it help to know that these BEVI cohorts are 
comprised of individuals from the United States and Central America as well as a highly internationalized 
group studying in Japan? From the standpoint of religion and gender, which of these three cultures comprised 
of students in higher education (i.e., not the population as a whole) would likely tend to be more 
“conservative” or “traditional” on average from the standpoint of “formative variables” (e.g., cultural norms)? 

If you answered Central America, you are correct. Specifically, Country/Region 1 is from project partner 
Dickinson College in the United States, and consists mainly of U.S. citizens (N = 158). Country/Region 2 is 
from project partner Georgetown University, which administered the BEVI to English-speaking students 
drawn from El Salvador and Honduras (N = 23). Country/Region 3 is from project partner Hiroshima 
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University, which administered the BEVI to an older, highly educated, and diverse cohort of English-speaking 
students which were studying in Japan (N = 24). 

Along similar lines, consider another BEVI Group Report analysis of U.S. citizens who have never 
studied abroad with a separate group of non-U.S. citizens/dual citizens who all completed the BEVI. 
Interestingly, on the majority of BEVI scales, these respondents - mostly college-aged students - showed 
similar scale results on most BEVI scales (i.e., within 5 points of each other). However, as Figure 12 
illustrates, on three of these scales, differences of over 10 percentage points emerged which would seem to 
make sense - and perhaps be predicted - in terms of the capacity of the BEVI to differentiate between these 
groups. Specifically, the “Non-U.S. Citizen/Dual Citizen” group showed the following BEVI scale elevations 
relative to the “U.S. Respondents who Have Not Studied Abroad”: Sociocultural Openness (75 versus 58); 
Ecological Resonance (65 versus 51); and Global Resonance (56 versus 45). At least on these comparisons, it 
appears that the BEVI is able to identify similarities and differences that are consistent with an intuitive 
understanding of what would be expected from the standpoint of validity. In short, on scales that are 
designed most directly to measure “other” and “larger world” concerns, it should not be surprising that the 
BEVI identified group-based differences that differentiate individuals who, on average, would more likely 
tend to be non-globally oriented at present (e.g., U.S. citizens who have not studied abroad) with those who 
likely have had more globally-oriented experiences overall relative to their U.S. counterparts (e.g., non- 
U.S./dual citizens). 


U.S. Citizens Who Nave Not Studied Abroad 


Non-U.S./Dual Citizens 


15. Sociocultural Openness 
16. Ecological Resonance 
17. Global Resonance 



15. Sociocultural Openness 
16. Ecological Resonance 
17. Global Resonance 


75 



56 


Figure 12. Comparisons between U.S. citizens who have not studied abroad and non-U.S. citizens/dual citizens. 


The point of these two juxtapositions - three Decile Profiles with relatively small sample sizes as well as 
larger sample sizes across U.S. and non-U.S./dual citizens - is not to suggest that such matters of cross-cultural 
usage are settled, only that - consistent with the processes through which it was developed and evaluated over 
many years - the BEVI 1) does appear able to identify within group differences across each of these cultures, 
illustrating the belief/value diversity therein; and 2) findings that emerged are generally consistent with what 
would be expected for participants drawn across different cultural and demographic configurations. Of 
course, as we continue to gather data across cultures and contexts, further analyses will be conducted at a 
number of levels (i.e., whether reliable differences and similarities emerge across different countries and 
regions). Such efforts are consistent with other big picture and longitudinal assessment initiatives, such as the 
exemplary World Values Survey (Inglehart, Basanez, & Moreno, 1998; Inglehart, Basanez, Dfez-Medrano, 
Halman, & Luijkx, 2004). 

Implication 10. As Kurt Lewin observed, there is nothing so practical as a good theory. The importance 
of thinking about why learning, growth, or development does — or does not — occur cannot be overemphasized. By 
specifying what our theoretical understandings and commitments are, and are not, we are compelled to contemplate 
what our underlying (and often implicit) models and epistemologies are, which bear directly on fundamental 
questions of assessment models and methods. Without considering such matters, we risk engaging in assessment 
research and practice that is face valid and superficial as well as lacking in substantive, practical, and real world 
impact, mainly because we are, perhaps inadvertently, minimizing or ignoring the complexity of these interacting 
phenomena. 

Although we rely on El Theory and the El Self in this project- mainly because they are integral to the 
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BEVI - the overarching point is that whatever assessment approach is pursued, the underlying theory should 
be specified in detail in order to explicate why particular research and practice methods or interventions are 
recommended and/or specific interpretations justified. In addition to providing guidance regarding why we 
contend that particular interventions should impact learning, growth, and development in particular ways, it 
simply is not possible to conduct high-level types of statistical analyses without a strong theoretical basis for 
doing so. For example, structural equation modeling requires us to specify the proposed theoretical 
relationship among various measurable constructs, and was integral to the illumination of various associations 
among formative variables (i.e., life history, ethnicity), mediators (i.e., various BEVI process-oriented scales), 
and outcomes (i.e., reported satisfaction upon the conclusion of a learning experience) reported throughout 
this article. Among other benefits, a strong theoretical basis for our measures and methods allows us to 

distinguish between descriptive and interpretive levels of analysis. That is, we need to examine both what 
constitutes change (e.g., how we know it is happening), a descriptive level of analysis, as well as why it is 
occurring (e.g., the rationale for why the changes we observe are in fact occurring), an interpretive level 
of analysis. We may be wrong or incomplete in our conclusions, but there can be no substitute for 
engaging in ongoing interpretation or explanation of why we observe the changes - or lack thereof - that 
we do (Shealy, in press). 

Implication 11. It is possible to identify the profile or signature of an institution or organization. The 

aggregate profile of the BEVI may illuminate how a student body at a given institution or organization tends to see 
self others, and the larger world. This profile appears to be relatively consistent across years, indicating that the 
characteristics of an institutions student body may be relatively stable and measurable. 

One of the more intriguing findings from the Forum BEVI Project was the emergence of unique BEVI 
“profiles” or “signatures” that appear to characterize the student body across various institutions or 
organizations. It is perhaps not surprising that different systems tend to attract different students. Such 
findings are in fact a central preoccupation of colleges and universities, which keenly are interested in a range 
of defining characteristics of their student bodies (e.g., entering GPA, SATs/ACTs, ethnicity, etc.). The fact 
that such differences extend beyond these standard indices, and seem to characterize deeper aspects of the 
aggregate psychological make-up of a student body, is a unique and thought-provoking insight. Consider for 
example Figure 13, which juxtaposes aggregate scores over a two year period (2010 and 2011) across all BEVI 
scales for a stratified random sample of over 600 incoming freshman students from Michigan State University 
(MSU). Such results illustrate at least two basic findings. First, MSU students demonstrate remarkable 
continuity on the BEVI across two separate years of administration. In addition to offering further support 
for the reliability of the measure, what is most intriguing is that the entering MSU student body seems to 
show a very high degree of consistency in how they experience self, others, and the larger world across 
different years. Such findings should be of interest at an administrative level, since the student body not only 
shows stability at the level of traditional indices (i.e., GPA), but also at the level of underlying life history and 
psychological variables, such as Emotional Attunement and Sociocultural Openness. Such findings are not 
mere abstractions, but bear significantly on the overarching “culture” or “feel” of a campus community. More 
importantly, they may illuminate areas for focus vis-a-vis student programming, communication, and 
engagement (e.g., in academic affairs, student affairs, etc.), and may help in the interpretation or 
understanding of various events and dynamics that occur on campus. For example, during one of the years of 
this study, a racially charged incident occurred at MSU. While the larger community (both on and off 
campus) engaged in much deliberation regarding whether or not the student body as a whole was “racist,” it 
was possible to observe - at least on the basis of these results - that it does not appear that the overall student 
body at MSU is inclined toward prejudicial beliefs; on the contrary, for the two years of this analysis, the 
entering MSU student body appears to be highly open toward and accepting of cultural differences (e.g., as 
illustrated by scores on Sociocultural Openness in the 71 st and 73 rd percentile in 2010 and 2011). Elowever, a 
more fine-grained analysis of BEVI data by decile (e.g., through a Decile Profile) illustrates that there may 
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well be small subsets of MSU students who are inclined toward racially prejudicial beliefs. Such findings not 
only put the matter in perspective vis-a-vis the student body as a whole, but may allow for more nuanced and 
targeted interventions, when appropriately designed and delivered. For example, it is quite common for 
colleges and universities to implement “sensitivity training” on various issues following incidents and/or as a 
proactive measure. Group based data of this sort may allow faculty, staff, and administrators to appraise more 
accurately the nature and scope of such problems rather than designing and implementing broad-based, one- 
size-fits-all interventions that may, inadvertently, do harm - rather than and/or in addition to - good (e.g., 
Tabit et ah, in press). 


Negative Life Events 
Need Fulfillment 
Identity Closure 
Basic Closedness 
Hard Structure 
Causal Closure 
Basic Determinism 
Divergent Determinism 
Socioreligious Closure 
Emotional Atlunement 
Positive Thinking 
Self Awareness 
Socioemotional Convergence 
Sociocultural Openness 
Socioreligious Traditionalism 
Gender Traditionalism 
Ecological Resonance 
Global Engagement 




10 20 30 40 


50 


2011 

■2010 


60 70 80 90 100 


Figure 13. Stratified random sample of two years of entering freshman students at Michigan State University across 
all BEVI scales. 


Second and consistent with the above observations, such results offer the possibility of understanding the 
aggregate make-up of the student body as a whole. More specifically, and focusing on a few key findings in 
Figure 13 across 2010 and 2011, MSU students overall appear not to report a high degree of unhappy life 
experiences prior to matriculation (Negative Life Events = 28 th percentile for both 2010 and 2011), and to be 
quite open/non-defensive generally (Basic Closedness = 20 th and 18 th percentile for 2010 and 2011). At the 
same time, the level of sophistication regarding “why humans do what they do” is not especially elevated at the 
outset of their higher education career (e.g., Basic Determinism = 71 st and 68 th percentile for 2010 and 2011), 
which is perhaps not surprising for students who are about to begin an intensive process of examining their 
underlying assumptions about the nature of reality. Auguring well over the long term in this regard, these 
students also evidence a high capacity for seeing the world in “shades of gray” rather than black and white (e.g., 
Socioemotional Convergence = 85 th and 86 th percentile for 2010 and 2011). Although such openness may not 
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necessarily extend to beliefs and values about gender roles (e.g., Gender Traditionalism = 80 th and 79 th percentile 
for 2010 and 2011), they appear very moderate overall regarding their religious inclinations (e.g., Socioreligious 
Traditionalism = 47 th and 42 nd percentile for 2010 and 2011). Moreover, they do appear highly open to different 
cultural beliefs and practices (e.g., Sociocultural Openness = 71 st and 73 rd percentile for 2010 and 2011) and 
are highly concerned about the environment and natural world (Ecological Resonance = 70 th and 71 st percentile 
for 2010 and 2011). Taken as a whole, such a profile offers important insights about the aggregate psychological 
makeup of the MSU student body, particularly when compared to the aggregate profile of institutions with 
quite different missions and traditions. In short, then, the generation of institutional or organizational profiles 
would seem to have at least five specific usages: 

1. Establishing a baseline for each cohort of entering students, which may be followed over time 
(through re-administration) in order to understand which aspects of the learning experience are 
associated with the most and least change. 

2. Examining interactions among demographic and background variables (e.g., income, 
acceptance rates), specific learning experiences (e.g., study abroad, participation in a learning 
community), and outcomes (e.g., BEVI scale changes, GPA). 

3. Developing and defining a “brand” (e.g., the relative openness of the student body), which 
allows for more precise strategic planning (e.g., “We wish to recruit more of X type of student 
in order to move the institution or organization in particular directions over time.”). 

4. Helping the institution or organization understand better who and what it is (e.g., what the 
normal distributional for the student body is as well as areas of greater and lesser variability). 

5. Moving beyond traditional demographic tracking by understanding deeper aspects of its 
identity as an institution or organization, including how the student body experiences self, 
others, and the larger world. 

Implication 12. Group reports help course instructors, program directors, and administrators 
understand better the nature of their particular class, cohort, or staff. Across a range of different settings and 
contexts, group reports have been used to help those in a position of leadership understand who their group is, which 
may help in planning and orientation purposes. Such reports also help assess what may be driving underlying 
processes of conflict or consensus for the group, which may result in more informed and sensitive leadership 
interventions. Further administration may allow for an examination of how the group sees itself others, and the 
larger world over the long-term after particular interventions and/or experiences occur. 

Transformational learning stands as an overarching goal of education writ large; yet the question needs to 
be asked: to what end and for what purpose (e.g., Cultivating the Globally Sustainable Self, 2015)? In cases 
where students will engage with vulnerable communities and complex topics, it is critical to establish the clear 
premise that student learning is not the only consideration; we also must ensure that students are engaging 
real issues and real people in a substantive manner. Understanding the signature profile of the institution 
generally - and the group profile of a cohort specifically - can assist instructors and program staff in making 
ethically sound decisions throughout the process of selection, preparation, monitoring, and support. 

One typical usage of the report system in this regard is to compare and contrast various subgroups within 
a larger institution or organization in order to understand how change occurs over time, while also 
appreciating the differences and similarities among such groups. Such analysis is important since even strong 
trends for a group as a whole should not obscure differences within the group as a number of comparative 
reports from this overall initiative reveal. Consider for example Figure 14, which contrasts incoming 
freshmen students at Michigan State University (MSU) with a selected group of student leaders, who are 
especially adept at addressing issues of diversity and multiculturalism (the “Diversity Group”). Presumably, if 
the BEVI is able to detect differences between groups both currently and across time, substantive differences 
should emerge in the profiles of these groups. In full recognition that we are examining different groups with 
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different Ns, a number of intriguing differences do exist and are worth considering as we contemplate future 
research. For example, there is a much higher degree of Negative Life Events amongst the Diversity group as 
compared to the Pre-entry group (Pre-entry = 28 th percentile; Diversity = 44 th percentile). Moreover, the 
Diversity group showed a much higher degree of attributional sophistication as evidenced by Basic 
Determinism (Pre-entry = 70 th percentile; Diversity = 30 th percentile) and a much lower degree of Gender 
Traditionalism (Pre-entry = 80 th percentile; Diversity = 42 nd percentile) as well as a higher degree of Self 
Awareness (Pre-entry = 52 nd percentile; Diversity = 69 th percentile) and Sociocultural Openness (Pre-entry = 
71 st percentile; Diversity = 83 rd percentile). The point is, although it is very relevant and informative to look 
at characteristics of the student body as a whole, it is no less important and illuminating to examine 
differences among subgroups within the broader population, both to understand differences and similarities 
between such groups and to map how various cohorts of students become “who they are” over time. 


Pre-entry Freshmen Students at Highly Diverse, Mostly Junior/Senior, 

Orientation (N=690) Undergraduate Leaders (N=38) 


1. Negative Life Events | 
2. Needs Closure I 


3. Identity Closure ^ 
4. Basic Closedness I 

5. Hard Structure ^ 

6. Causal Closure | 
7. Basic Determinism I 

8. Divergent Determinism I 

9. Socioreligious Closure 

10. Emotional Attunement | 
11. Positive Thinking I 
12. Self Awareness I 
13. Socioemotional Convergence | 
14. Sociocultural Openness | 
15. Socioreligious Traditionalism I 
16. Gender Traditionalism I 


17. Ecological Resonance | 
18. Global Engagement | 


71 


High 


71 


55 


Low 


1. Negative Life Events 
2. Needs Closure 
3. Identity Closure 
4. Basic Closedness 

5. Hard Structure 

6. Causal Closure 
7. Basic Determinism 

8. Divergent Determinism 

9. Socioreligious Closure 
10. Emotional Attunement 

11. Positive Thinking 
12. Self Awareness 
13. Socioemotional Convergence 
14. Sociocultural Openness 
15. Socioreligious Traditionalism 

16. Gender Traditionalism 

17. Ecological Resonance 
18. Global Engagement 



High 


Figure 14. BEVI organizational profiles between two cohorts of students within the same institution. 
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Another area of usage is to understand the similarities and differences between institutions. Consider for 
example Figure 15, which contrasts two BEVI profdes from two project partners: Dickinson College is a 
highly internationalized, majority female, liberal arts institution; Michigan Technological University (MTU) is 
not especially internationalized, is majority male, and has a strong engineering focus. 


Freshmen at Highly Internationalized 
Liberal Arts College (N=157) 


Freshmen at a Technical University 
with Engineering Focus (N=292) 


Low 


1. Negative Life Events 

2. Needs Closure 

3. Identity Closure 

4. Basic Closedness 

5. Hard Structure 

6. Causal Closure 

7. Basic Determinism 

8. Divergent Determinism 

9. Socioreligious Closure 

10. Emotional Attunement 

11. Positive Thinking 

12. Self Awareness 

13. Socioemotional Convergence 

14. Sociocultural Openness 

15. Socioreligious Traditionalism 

16. Gender Traditionalism 

17. Ecological Resonance 

18. Global Engagement 



High 


Low 


High 


1. Negative Life Events 
2. Needs Closure 
3. Identity Closure 
4. Basic Closedness 

5. Hard Structure 

6. Causal Closure 
7. Basic Determinism 

8. Divergent Determmism 

9. Socioreligious Closure 
10. Emotional Attunement 

11. Positive Thinking 
12. Self Awareness 
13. Socioemotional Convergence 
14. Sociocultural Openness 
15. Socioreligious Traditionalism 

16. Gender Traditionalism 

17. Ecological Resonance 
18. Global Engagement 



Figure 15. BEVI organizational profiles between two cohorts of students across different institutions. 
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What do we conclude from these contrasting profiles? Among other notable distinctions, observe the 
differences on scales measuring the relative attributional complexity across the two institutions, as evidenced 
by percentile scores on Causal Closure (Dickinson = 37th percentile; MTU = 78th percentile) and Basic 
Determinism (Dickinson = 35th percentile; MTU = 75th percentile). From an interpretive standpoint, 
Dickinson “first years” from this sample at least evince a much greater tendency to recognize the complexity 
inherent in making attributions about self, others, and the larger world (i.e., they are much less likely to 
endorse black and white statements regarding why human beings are the way they are and/or why events 
happen as they do). To highlight a couple of other examples, Dickinson students from this sample are much 
more likely than MTU students to focus upon “inner” thoughts and experiences in self and other (Self 
Awareness: Dickinson = 65th percentile; MTU = 33rd percentile) and much less likely to endorse traditional 
notions of what it means to be “female” or “male” (Gender Traditionalism: Dickinson = 50th percentile; 
MTU = 78th percentile). Nothing in such a comparison is intended to illustrate or imply that one group of 
students is “better” than any other, the point is rather that we must understand our student bodies in order to 
appreciate where we are now (i.e., what our baseline is) and where we wish to go. Without such information, 
it is difficult to ensure that our instructional and administrative goals, methods, and plans are realistic, 
targeted, and consistent with overarching aspects of our student body and institutional mission. 

Such assessment possibilities are aided further by another dimension of the reporting system, which 
distinguishes among “high,” “medium,” and “low” full scale scores across subgroups, as described above. 
Various indices and tables on the reporting system address this issue, including Background-Domain 
Contrast, which illustrates how different or similar the group is at the level of background information and 
domain scores by the lowest 30 percent, middle 40 percent, and highest 30 percent of full scale scores. 
Consider the following excerpts of Time 1/Time 2 Group Report results from a sample of participants (N = 
101) on Background Domain Contrast. 

Of other highlights that might be emphasized, note from Tables 11 and 12 that individuals who, at the 
outset of a learning experience, are at the low, middle, or high end of the “full scale” continuum demonstrate 
specific profile characteristics, both in terms of “background” variables (i.e., age, gender, education) and 
“domain” variables (i.e., one of the seven areas under which BEVI scales are clustered). In particular, for this 
sample of students, those most inclined toward an international, multicultural, or transformative learning 
experience are: 

• older (average age = 27 versus 19); 

• approximately twice as likely to be female (ratio of females to males of 20:11 versus 16:15); 

• more likely to be non-Caucasian than Caucasian (ratio of non-Caucasian to Caucasian of 11:20 
versus 6:25); 

• have more years of post-secondary education (7 years versus 2 years); 

• more likely to come from a less wealthy family ($70,000 versus $85,000); and, 

• on the domain section of Background-Domain Contrast (comprised of specific BEVI scales), 
show substantial differences across the seven domains (e.g., 78 versus 32 on the domain of 
Global Access). 
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Table 11 


Sample Findings from Background Information Section of Background-Domain Contrast (N=101) 


Lowest Full Scale 

Middle Full Scale 

Highest Full Scale 

Mean = 41 

Mean = 58 

Mean = 71 

Background Information 

Background Information 

Background Information 

Average Age: 19 

Average Age: 22 

Average Age: 27 

Males: 15 

Males: 11 

Males: 11 

Females: 16 

Females: 26 

Females: 20 

Caucasians: 25 

Caucasians: 26 

Caucasians: 20 

Non-Caucasians: 6 

Non-Caucasians: 11 

Non-Caucasians: 11 

► Details: 

► Details: 

► Details: 

Raised in U.S.: 31 

Raised in U.S.: 37 

Raised in U.S.: 29 

Not Raised m U.S.: 0 

Not Raised in U.S.: 0 

Not Raised in U.S.: 2 

► Details: 

► Details: 

► Details: 

Average Years of Education: 2 

Average Years of Education: 3 

Average Years of Education: 7 

Average Parental / Household Income: S85,484 

Average Parental / Household Income: S77,162 

Average Parental / Household Income: S70,161 
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Table 12 

Sample Findings from Domain Section of Background-Domain Contrast 


n. Full Scale: 41 

H. Full Scale: 58 

H. Full Scale: 71 

III. Formative Variables 

in. Formative Variables 

III. Formative Variables 

• Negative Life Events: 40 

• See Background Information 

• Negative Life Events: 64 

• See Background Information 

• Negative Life Events: 51 

• See Background Information 

IV. Fulfillment of Core Needs: 37 

IV. Fulfillment of Core Needs: 69 

IV. Fulfillment of Core Needs: 82 

V. Tolerance of Disequilibrium: 36 

V. Tolerance of Disequilibrium: 52 

V. Tolerance of Disequilibrium: 68 

VI. Critical Thinking: 35 

VI. Critical Thinking: 51 

VI. Critical Thinking: 73 

VH. Self Access: 57 

VH. Self Access: 76 

vn. Self Access: 78 

Vin. Other Access: 30 

VIE. Other Access: 55 

VIE. Other Access: 78 

IX: Global Access: 32 

IX: Global Access: 54 

IX: Global Access: 78 


From the standpoint of the report system, then, the implications of such findings should be clear. First, 
differences within groups should not be underestimated but expected (i.e., we should assume that groups may 
well have important differences both at the level of background and how they experience self, others, and the 
larger world). Second, for those who are leading learning, growth, or development experiences, reviewing 
such differences before a course, program, or workshop begins may be very helpful in understanding the 
nature of the group with which they will be interacting throughout the experience. Third, for appropriately 
trained and skilled facilitators, it may be helpful to share some or all results with participants in order to 
sensitize them to such differences and similarities among them (i.e., to highlight the fact that there may be 
very good reasons for why participants react as they do to the experiences they are about to have, as illustrated 
on several occasions through various projects reported in this article). Fourth and finally, such findings may 
help at the level of interpretation (i.e., the role of the instructor and/or learning experience should be 
accurately appraised in terms of relative impact) since, most likely, there will be multiple interactions 
occurring simultaneously that affect learning processes and outcomes. In short, by reviewing such report- 
based results, more reflective, aware, and skillful learning interventions may be developed, implemented, and 
appraised since we have a basis for understanding why learning processes and outcomes unfold as they do. 

Implication 13. By jointly utilizing individual and group reports, multiple opportunities for self- 
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assessment, enhanced understanding, and group development may be facilitated. On many occasions, different 
cohorts have used the individual and group report system simultaneously in order to enhance understanding of how 
participants see themselves, others, and the larger world. By juxtaposing individual reports (which are personalized 
based upon unique scores, and read privately by each individual) with group reports (which show similarities and 
differences within the group, and are reviewed publicly with the group as a whole), abundant opportunities arise for 
dialogue, engagement, and growth. 

Data from the Forum BEVI Project strongly suggest that the sorts of international, multicultural, and 
transformative experiences examined in this project do influence the amount and degree of growth, 
development, and learning that occur. But to illustrate such phenomena in real time to administrators, 
program directors, course/workshop instructors, students, and others who may have strong interests in such 
matters - but may not have a statistical background - various aspects of the report system are of particular 
relevance (i.e., to show in clear terms whether and how such “changes” actually occur). Because assessment 
results from various measures often are not communicated in a way that translates readily into real world 
usage, the Working Group devoted particular attention to how the BEVTs report system should be used from 
the standpoint of accessibility, relevance, and best practice. Consistent with our original charge, we were 
interested in understanding how the translation of data/scores into accessible report-based systems could be 
used to promote growth and learning in general, which should have implications for instrument-based report 
systems including, but not limited to, the BEVI. As such, considerable effort has been devoted to trying to 
understand how such a “report system” might best translate otherwise abstruse statistical findings into usable 
findings. In particular, considerable deliberation focused on how feedback from the individual reports of the 
BEVI should be given to students or staff. Among other conclusions, it was determined that a brief 
orientation by a trained BEVI administrator was deemed important in order to provide sufficient context for 
recipients. Moreover, given the comprehensive nature of BEVI reports, it proved important that those in 
leadership or administrative positions were sufficiently well-versed in matters of interpretation. That is 
because incomplete or superficial understanding of scales or indexes is contraindicated, especially when 
contemplating real world implications or applications based upon test scores or reports (e.g., Braskamp et ah, 
2012). In light of such considerations, it may be helpful to provide additional context prior to presenting 
actual sample reports that emerged during this project. 

Essentially, the BEVI produces reports that may be used at three levels: individual, group, and 
organization. 21 Individual reports are developed for each participant based upon their unique responses to the 
background information section of the BEVI along with their scores on specific BEVI scales. Depending 
upon their responses, a report is generated under the following headings, which correspond with the 
theoretical and empirical structure of the BEVI: 

1. Introduction: Provides an overview of the BEVI. 

2. The Foundation: ‘Formative Variables’ and ‘Core Needs’: Provides an indication of what 
the respondent reports about their own life history relative to others. 

3. Tolerance of Disequilibrium: Describes whether the respondent sees him or herself as “very 
clear” or “not sure” about who they and others “are” under the auspices of how “confident” 
or “questioning” the respondent appears to be. 

4. Making Sense of Why We Do What We Do: Indicates attributional tendencies in general 
(e.g., how and why people do what they do and why events happen as they do). 

5. Access to Yourself and Your Thoughts, Feelings, and Needs: Describes how the 
individual deals with their own emotions as well as their interest in and predilection towards 
introspection and reflection upon “the self.” 

6. Access to the Thoughts and Feelings of Others: Describes how the person tends to regard 


This section and selected others in this article are adapted and/or excerpted from the BEVI test manual. 
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and experience issues that are of consequence at a sociocultural level (e.g., beliefs about 
politics, religion, gender, or the way society “should be structured”). 

7. Access to the Larger World: Indicates one’s perspectives on “big picture” issues of the 
environment (e.g., the degree to which one is or is not concerned about ecological issues) and 
global engagement (e.g., the degree to which we should or should not be concerned about, or 
invested in, what is happening outside of our own country, culture, and context). 

8. Conclusion: Provides context for the above report and offers closing thoughts to consider. 

Individual reports are designed for application in a range of contexts and settings (e.g., as a teaching 
tool in classes; pre-departure orientation for an international learning experience; to promote growth and 
development in organizational and other applied contexts), and are intended to facilitate thoughtful and 
substantive reflection on self, others, and the world at large. 

Group/Organizational reports. Basically, there are two types of group/organizational reports. First, 
Time 1 Group Reports describe results from the original administration of the BEVI to a specific group; 
secondly, Time 1/Time 2 Group Reports examine how groups change from one administration of the BEVI to 
the next (Time 1/Time 2/Time 3, etc. reports also may be derived, and are recommended in terms of tracking 
trends over the long term, as noted above). The primary difference between Time 1 and Time 1/Time 2 
reports are a few indices that may be developed only on the basis of comparisons between groups from Time 1 
to Time 2 (e.g., Worldview Shift). Each type of report includes a series of tables and indices that provide 
information about specific aspects of the group that completed the BEVI. For example, Background-Domain 
Contrast illustrates how different or similar the group is at the level of background information and domain 
scores by the lowest 30 percent, middle 40 percent, and highest 30 percent of full scale scores. As another 
example, Profile Contrast illustrates how different and similar the group is across all 17 BEVI scales via the 
lowest 30 percent, middle 40 percent, and highest 30 percent of full scale scores. 

Mindful of the overarching goals of relevance and impact of such report-based assessment, including but 
not limited to the BEVI, questions that emerged consistently during our work with this measure (i.e., those 
that proved useful in facilitating interpretation and group-based discussion) included: 

1. How would you describe the group, organization, or institution overall in terms of the most 
striking findings? 

2. How is the group most different and most similar to itself? 

3. Which findings are most surprising, and how can you make sense of such results on the basis of 
data that are provided? 

4. Which findings were expected, and what do they suggest about your group, organization, or 
institution? 

5. Which findings would be most relevant or interesting to a particular group or your overall 
organization/institution, and why? 

6. How could findings inform or shape group-based discussions and processes? 

7. What do findings suggest in terms of a particular course, program, or learning experience (e.g., 
Where might areas of consensus or conflict occur over time? How might patterns interact with 
a learning, growth, or development experience in terms of overarching goals?)? 

8. What might findings suggest about the effectiveness of a particular learning, growth, or 
development experience? 

9. How might findings be used to facilitate the development of new or improved learning, 
growth, or development experiences? 

10. How might findings be used to track changes over time? 
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There are many other questions to be asked of such report results. Overall, we learned that it was 
maximally important for users to take sufficient time to review and reflect upon reports in order to ascertain 
which tables, indices, or specific findings are most relevant for their specific goals and purposes. 

Over the course of this project, the most common usage was a combination of group based and 
individual reports (i.e., during a pre-departure study abroad orientation program; at the outset of a 
multicultural course or workshop; in a residence hall at the beginning of the academic year). In this case, 
individual students would be offered the opportunity to receive their own individual report, which they (and 
only they) would read. Then, a group report that had been developed on the basis of the aggregate scores 
from this group of individuals would be shared with the group. In this way, individuals are able to reflect (in 
private) upon their own BEVI results with the indices and other scale information from the BEVI group 
report (in public). This dialectic process between private and public reflection - which allows for safe internal 
dialogue and voluntary external dialogue - seems to be integral to the sort of reflection, awareness, and insight 
that students and staff frequently reported from this process (e.g., “It helped to think about why I am who I 
am”; “It helps me appreciate why others believe what they do”). 

At another level, the Group Reports help program directors or course instructors understand better the 
differences and similarities of history or worldviews by the members of their group. For example, it was not 
uncommon for bimodal distributions on various scales (e.g., Gender Traditionalism; Socioreligious 
Traditionalism) to emerge, which when combined with the breakdown of scale scores by profile, permit the 
leader or administrator of a learning experience to understand better the makeup of a group even before 
departing and/or at the outset of a class. From an applied standpoint, such results may be used productively 
by showing group-based results and asking for volunteers to discuss “why they believe what they believe,” a 
process that can lead to rich dialogue and reflection by participants (e.g., Iyer, 2013; Shealy, in press). 

One common usage of the BEVI report system during this project was to engage students in an 
examination of the interface between their individual reports and the group report. To illustrate this process, 
we focus upon a learning community in one of our project partners, James Madison University, called 
Madison International, which brings together U.S. and international students as part of a living and learning 
community who complete coursework and participate in other experiences together during their first year at 
the university (see http://www.jmu.edu/international/isss/students/get-involved/mi/index.shtml) . The 
example below of individual/group report usage focuses on a subset of these students (N = 22), who 
completed a course called Making Sense of Beliefs and Values: A Guided Tour for Global Citizens. After 
providing consent and completing the BEVI, individual reports were provided to each student in the course. 
The first page of a sample individual report is shown in Figure 16. 

The BEVI trained coordinator then reviews the Group Report with all members of the cohort. 
Depending on the goals of a particular BEVI workshop, different sections of the report may be emphasized. 

In the case of Madison International, most of the focus centered on three components of the Group Report: 

1) Background Information (i.e., descriptive information regarding how the group “breaks down” across a 
range of different demographic variables); 2) Aggregate Profile (i.e., the aggregate scores for participants across 
all BEVI scales); and 3) Decile Profile (i.e., the aggregate scores across each scale in increments of 10 percent, 
in order to illustrate how the members of a group are dispersed across each of the BEVI scales). Figures 17 — 

19 illustrate these three components of the BEVI Group Report for the Madison International Learning 
Community. 
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Beliefs, Events, and Values Inventory 


You and Your Worldview 

A Personal Report from the 
Beliefs, Events, and Values Inventory (BEVI)™ 

User: 9389488 Date of Test: 8/29/2012 

The Beliefs, Events, and Values Inventory (BEVI) is designed to help people gain greater awareness of then own beliefs and 
values. The BEVI also examines how r beliefs and values may influence learning, personal growth, relationships, group or 
organizational processes, and the pursuit of life goals. In addressing these issues, this report contains information about how you 
see yourself, others, and the larger world. However, because the BEVI takes no position at all on whether the answers you 
provided are right, wrong, true, or false, neither does this report make any such claims. Instead, this report is designed to promote 
reflection about issues in life that often are relevant to us all. If you have any questions about the BEVI, please contact the BEVI 
administrator at your college, university, or study abroad provider. You also may contact us directly at ibavi@ibavi.ora . Before 
reading the following report, you must agree to the three conditions below either by clicking the Agree button or by signing a 
separate agreement form that will be provided to you prior to reading the report: 

1. You (9389488) voluntarily' completed the BEVI and subsequently requested this follow' up report, w'hich y'ou are reading 
voluntarily (i.e., you agree that no person, group, organization, or entity' is requiring you to read this report): 

2. Every effort has been made to communicate information from the BEVI in a way' that is helpful and constructive. How'ever, if 
you experience distress upon reading this report, y'ou agree to contact the BEVI administrator m your organization, the 
coordinator of a BEVI feedback session, and'or a health or mental health care provider. 

3. You are free to keep a copy' of this report for yourself. However, because the BEVI and this report are copyrighted, you agree 
not to photocopy this report or distribute it (electronically or otherwise) to anyone else. 


Introduction 

Pick up a newspaper or turn on a radio or television anywhere m the w'orld and one fact becomes immediately' clear: beliefs and 
values are integral to the human experience. People have strong opinions about topics from politics, religion, education and the 
arts, to marriage, family, gender, and sexuality. However, whether a specific belief is "right" or "wTong" is arguably less 
important than understanding the complex interaction among thoughts, feelings, behavior, life history', and context that results in 


Figure 16. Sample introductory page from the BEVI individual report. 
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Beliefs, Events, and Values Inventory 


BEVI Group Report 

Madison International 


N = 22 


Background 


Ethnic Group 

Black And White 

4.55% 

Black African-American 13.64% 

Hispanic Latino 

9.09% 

Scandinavian 

4.55% 

White 

68.18% 


Gender 

Female 63.64% 

Male 36.36% 

Political Orientation 


Democrat 

31.82% 

Independent 

31.82% 

Philosophical Anarchist 

4.55% 

Republican 

27.27% 

Social Democrat (mostly Democratic) 

4.55% 


Religious Orientation 

Figure 17. Introductory excerpt from the background information section of the group report for the Madison 
International Learning Community. 
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Negative Life Events 
Need Closure 
Identity Closure 
Basie Closedness 
Hard Structure 
Causal Closure 
Basic Determinism 
Divergent Determinism 
Socioreligious Closure 
Emotional Attunement 
Positive Thinking 
Self Awareness 
Socioemotional Convergence 
Sociocultural Openness 
Socioreligious Traditionalism 
Gender Traditionalism 
Ecological Resonance 
Global Engagement 
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Figure 18. Aggregate Profile of the BEVI for the Madison International Learning Community. 
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Deciles: 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1. Negative Life Events 

18% 

14% 

23% 

5% 

9% 

0% 

14% 

9% 

5% 

5% 

2. Needs Closure 

23% 

14% 

23% 

18% 

23% 

0% 

0% 

0% 

0% 

0% 

3. Identity Closure 

23% 

5% 

5% 

23% 

36% 

5% 

0% 

0% 

0% 

5% 

4. Basic Closedness 

45% 

9% 

5% 

23% 

14% 

5% 

0% 

0% 

0% 

0% 

5. Hard Structure 

32% 

18% 

5% 

9% 

18% 

0% 

9% 

5% 

5% 

0% 

6. Causal Closure 

9% 

0% 

18% 

0% 

5% 

27% 

14% 

0% 

9% 

18% 

7. Basic Determinism 

5% 

9% 

18% 

14% 

9% 

9% 

18% 

0% 

5% 

14% 

8. Divergent Determinism 

32% 

18% 

32% 

5% 

5% 

5% 

0% 

0% 

0% 

5% 

9. Socioreligious Closure 

18% 

5% 

9% 

5% 

5% 

18% 

5% 

9% 

23% 

5% 

10. Emotional Attunement 

0% 

0% 

5% 

0% 

5% 

14% 

5% 

18% 

27% 

27% 

1 1. Positive Thinking 

5% 

0% 

9% 

9% 

9% 

9% 

9% 

9% 

14% 

27% 

12. Self Awareness 

14% 

9% 

0% 

9% 

5% 

5% 

0% 

9% 

14% 

36% 

13. Socioemotional Convergence 

0% 

0% 

0% 

0% 

0% 

5% 

9% 

18% 

27% 

41% 

14. Sociocultural Openness 

0% 

0% 

0% 

0% 

0% 

14% 

14% 

23% 

32% 

18% 

15. Socioreligious Traditionalism 

23% 

5% 

9 % 

14% 

0% 

0% 

0% 

32% 

9% 

9% 

16. Gender Traditionalism 

5% 

5% 

9 % 

0% 

14% 

9% 

9 % 

14% 

23% 

14% 

17. Ecological Resonance 

0% 

0% 

0% 

0% 

5% 

27% 

9% 

14% 

14% 

32% 

18. Global Engagement 

5% 

9% 

0% 

14% 

5% 

5% 

14% 

9% 

18% 

23% 

Deciles: 

1 


•5 

4 

5 

6 

7 

8 

9 

10 


Figure 19. Decile Profile of the BEVI for the Madison International Learning Community. 

How were the above three components of the BEVI group profile used with the Madison International 
Learning Community? After reviewing the background characteristics of the group (e.g., highlighting areas of 
similarity and difference, from Figure 17), the bulk of this session focused on explaining and discussing the 
results from the Aggregate Profile and Decile Profile. To highlight a number of these scales, and among other 
findings that might be emphasized, note overall that the group as a whole (from the Aggregate Profile in 
Figure 18): 

• reported relatively positive life histories (e.g., low Negative Life Events, Needs Closure); 

• is quite open to basic thoughts/feelings that characterize the typical experience for most human 
beings (e.g., low Basic Closedness); 

• demonstrates a moderate degree of attributional complexity regarding why human beings do 
what they do and why events in the world happen as they do (e.g., moderate Basic 
Determinism); 

• evidences a moderate degree of religiosity (e.g., moderate Socioreligious Traditionalism); 

• is highly self and emotionally aware (e.g., high Emotional Attunement, Self Awareness); 

• appears to possess a high capacity to experience self and others in shades of gray rather than in 
dichotomous terms (e.g., high Socioemotional Convergence); 

• is very open to cultural beliefs and practices that are different from their own (e.g., high 
Sociocultural Openness); 

• is concerned about the environment and natural world (e.g., high Ecological Resonance); and, 

• is deeply interested in making a difference in the world (e.g., high Global Engagement). 
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In many ways, such a profile would perhaps be consistent with a group of individuals who have self- 
selected to be part of an international living and learning community. However, as noted throughout this 
article, it is very important to go beyond aggregated results in order to understand areas of difference and 
similarity within the group as a whole. To do so, let’s focus on one of the BEVI scales, Socioreligious 
Traditionalism, from the Decile Profile (Figure 19), particularly because this scale promoted a great deal of 
reflection and discussion by community members. As illustrated below in Figure 20, note first the 
extraordinary dispersion by the group across this specific scale from Decile Profile. 


15. Socioreligious Traditionalism 

23% 

5% 

9% 

14% 

0% 

0% 

0% 

32% 

9% 

9% 


Figure 20. The Socioreligious Traditionalism scale from the Decile Profile for the Madison International Learning 
Community. 


In particular, approximately 50 percent of the learning community falls between the 1 st and 40 th 
percentile of the scale, whereas the remaining 50 percent falls between the 70 th and 100 th percentile (no 
respondents fall between the 41 st and 69 th percentile). Note further that approximately one quarter of the 
community occupies the lowest 10 th percentile on this scale (i.e., is essentially disavowing religion and 
religious belief). The presentation of this finding sparked a great deal of dialogue by the group. From a 
thematic perspective, the group seemed fascinated by the fact that such profound differences existed among 
individual members. They also discussed larger issues, such as the purpose of existence, where beliefs and 
values come from, and why such questions should - or should not - matter to us as individuals and as a 
society. In short, the juxtaposition of individual and group reports offers a model for how the individual 
learner may have a private encounter with “self” regarding one’s own worldview, to be followed by a skillfully 
facilitated dialogue on how the larger group sees self, others, and the larger world. Such a process may well 
promote a great deal of learning, growth, and development in a manner that is consistent with larger course, 
program, and institutional goals (Iyer, 2013). 

Implication 14. Changes in beliefs and values about self, others, and the world at large appear 
determined largely by the 7Ds (duration, difference, depth, determine, design, deliver, debrief). The odds of 
facilitating effective learning, development, and growth opportunities are enhanced considerably by attending to the 
7Ds before, during, and after a learning intervention is developed, implemented, evaluated, and refined. 

As noted throughout this article, to understand and assess the processes and outcomes of international, 
multicultural, and transformative learning, it is crucial both to measure and account for the impact of within- 
group differences on the learning processes and outcomes we observe. 

Specifically, the “7Ds” (Shealy, in press) include the following variables: 

1. duration (i.e., how long an international, multicultural, or transformative education experience 
occurs); 

2. difference (i.e., how different the experience is from what the “self’ of the experiencer is 
accustomed); 

3. depth (i.e., what the capacity of the learner is to experience all that the intervention is able to 
convey); 

4. determine (i.e., through formal and informal assessment, how well the intervener understands 
his/her audience); 

5. design (i.e., based upon knowledge of the audience and careful deliberation and development, 
what the quality of the intervention is); 

6. deliver (i.e., how well the intervener can fulfill the transformative potential of the 
intervention); and, 
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7. debrief (i.e., before, during, and after the intervention, how deeply the intervener assesses the 
nature of the learning experience, and uses such feedback to improve planned, current, and 
future interventions). 


Although the 7Ds are based upon a combination of data-driven findings and best practice 
recommendations, future research could examine specific aspects of each in greater detail. For example, it 
would be possible to evaluate empirically how a relative understanding of one’s audience is associated both 
with various characteristics of the audience (e.g., from BEVI group scores) as well as differences in learning 
outcomes (e.g., to what degree congruency between the intervener and the audience is consistent with better 
learning outcomes, such as higher grades or greater satisfaction). In short, the 7Ds are offered as data/practice 
based hypotheses that may help in the generation of specific questions that are worthy of further investigation. 

Implication 15. Best practice in assessment requires best practice in research. By attending to data-based 
findings regarding the mediators and moderators of teaching, training, and learning — and adopting mixed methods 
approaches to assessment that account for the complexity of the variables at hand — we are much better able to plan 
for, implement, and evaluate the effectiveness of our interventions. Ultimately, our goals are to: 1) understand the 
complex interactions between teachers, trainers, and learners as well as the complex processes and outcomes of 
learning, growth, and development, and 2) use such information to enhance the quality and impact of teaching, 
training, and learning in the real world. 

This fifteenth and final implication really is a take-home message from the entire Forum BEVI Project. 
As envisioned in the original charge to the Working Group, we knew that our ability to hone in on very 
complex and interacting processes of learning, growth, and development would require exquisite attention to 
myriad issues over a sustained period of time. Several of these “lessons learned” were codified in a symposium 
delivered at the Association for International Education Administrators (AIEA) in the fourth year of this 
project (Gerber, Gottlieb, Pysarchik, & Shealy, 2011), which sought to summarize best practice 
recommendations from four big picture assessment initiatives, including the Forum BEVI Project. These 
include the following six recommendations: 

1. Assess Deeply (e.g., encourage innovation; integrate mixed methods; strive for ecological 
validity; evaluate interactions, mediators, and levels of analysis); 

2. Address Challenges (e.g., political, time, economic; science/humanities split; not knowing 
what one doesn’t know); 

3. Secure Support (e.g., institutional/organizational; develop functional team; identify and pool 
resources); 

4. Ensure Expertise (e.g., theory/data link; research design; psychometrics; statistical); 

5. Develop Plan (e.g., specify assessment goals and outcomes; select measures; assign roles and 
responsibilities; monitor/correct processes); and 

6. Value Collaboration (e.g., recognize team talents; appraise/meet local needs; facilitate buy in; 
solicit input/feedback). 

From our perspective, these six recommendations provide a sense of what is necessary to design, 
implement, and evaluate a multi-site, multi-year assessment project in a manner that is more likely than not 
to generate outcomes of substance and impact. More than anything, beyond the letter of such 
recommendations, their spirit requires the cultivation of patience, kindness, and humor. That is because the 
one constant we encountered throughout was change - at the level of direction, approach, method, 
interpretation, partners, technique, administration, analyses, logistics, and so forth. Thus, there can be no 
substitute for relying on relational connections and good will toward one another, established through 
countless phone calls, emails, meetings, and presentations, over many years of collaborative work together. 
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Forum BEVI Project: Summary and Conclusion 

We began this article by offering a big picture perspective regarding the w hat and why of 
internationalization within the field of higher education. We turned then to an overview of contemporary 
efforts in the assessment of international, multicultural, and transformative learning. We next observed that 
the complex variables that mediate and moderate global learning outcomes still await precise identification. 

In particular, it behooves scholars, educators, and administrators to recognize and account for the profound 
and underlying human complexities - affective, behavioral, cognitive, contextual, cultural, developmental, 
historical - that interact with a particular international, multicultural, and transformative learning experience 
to produce the processes and outcomes that we ultimately observe. We then described the partnership formed 
in 2007 between the Forum on Education Abroad and International Beliefs and Values Institute, summarizing 
the goals and activities of the Forum BEVI Project Working Group. We provided a brief overview of the 
Beliefs, Events, and Values Inventory and its Equilintegration or El theoretical framework. We presented 
specific applications of the BEVI throughout this initiative, and then provided an overview of the analyses and 
sample characteristics that informed our work. Finally, we discussed the 15 implications in detail. 

We recognize fully that the above discussion is, to say the least, comprehensive and intensive, which 
befits a six year initiative involving multiple institutions and organizations as well as hundreds of analyses and 
thousands of participants. Ultimately, it is our hope that this project and its fifteen implications contribute to 
1) a deeper understanding of the complex interactions and fascinating processes that are associated with who 
learns what and why, and under what circumstances as well as 2) greater appreciation for the demonstrable 
fact that such complexities can, and must, be assessed with depth, persistence, and rigor. In this regard, 
throughout our work, it has been heartening to experience our collective sophistication grow - theoretically, 
methodologically, empirically, and in applied terms - with attendant and constructive impacts upon a range of 
programs, courses, activities, and processes in the real world. From our perspective, the sine qua non of such 
work has been realized in the “aha” moments we all have shared. Formerly abstruse dynamics suddenly have 
been illuminated when we observe, for example, that conflicts or issues in a group actually are revealed via a 
bimodal distribution on specific scales, which indicate vast differences - or remarkable similarities - among 
cohort members in how and why they experience self, others, and the larger world as they do, with 
concomitant implications for group relations and learning processes over the short- and long-term. Likewise, 
we no longer get many questions like, “Why are you asking about life events?” as we have shown and learned 
together that the perception of one’s own life history may change, often substantially, following exposure to 
cultures and contexts that are significantly different from one’s own. In short, it seems increasingly to be 
understood - on the basis of theoretical framing, empirical data, and real world interventions - that who we 
are prior to a learning experience must be taken into account if we are to understand how and why the “7Ds” 
interact to produce the learning processes or outcomes that we do or do not observe, including (but not 
limited to) 1) the learning experiences we create and deliver; 2) what people bring to the learning experience 
based upon their life histories and worldviews; 3) how we’re built and what we need as human beings; and 4) 
the ubiquitous interactions among these and related levels of analysis. Such findings seem well illuminated by 
the El framework - that we inexorably are built to create a “version of reality” in the service of our core needs, 
the formative variables that shape and define us, and the subsequent beliefs and values that we call our own - 
which further influence how and why we experience self, others, and the larger world as we do (e.g., Shealy, in 
press; Shealy et ah, 2012). 

As scholars, educators, professional staff, and administrators, we now have a better basis for 
understanding how our work interacts with the lives and worldviews of those who come before us seeking to 
learn, grow, and change, even if they - and we - may not apprehend yet what the implications are for them, 
and us, of engaging in these immersion processes so closely together. The fact that we co-create meaning 
through our shared lives and experiences is a substantive conclusion of this project that warrants further 
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exploration, as we seek to hone in that much more on the interactions that ultimately are associated with who 
learns what and why, and under what circumstances (e.g., Cultivating the Globally Sustainable Self, 2015). In 
short, we must take our responsibilities very seriously, not only because that’s “the right thing to do,” but 
because abundant evidence from this project suggests that we may differ substantially in our capacity and 
inclination to carry out these important obligations as facilitators of learning, growth, and development. 

From our perspective, such an empirical reality is not, and should not be, an impetus for anxiety or doubt, but 
rather an opportunity to engage in a deeper exploration of how we may better fulfill our potential as 
educators, program developers, administrators, and scholars. Just as we expect our students to learn and grow, 
findings from this project indicate that we can and should expect no less of ourselves, even as we pursue these 
means and ends with a steadfast appreciation that we never are “done” vis-a-vis our own personal and 
professional development. 

In the final analysis, what has been most gratifying for all of us over the years is to hear so often that 
what we have been trying to understand and explicate through this initiative has in fact been impactful, 
meaningful, and relevant. Through our work, we have come to see more clearly that we are - ineluctably - 
change agents who are granted the privilege of transforming the learners who enter our educational institutions 
and organizations, and are in turn, transformed ourselves. Since we are charged to be agents of change by the 
next generation of citizens - and compelled by the pressing needs and opportunities of our increasingly 
interdependent world - it behooves us to answer this call in a manner that is optimally informed and 
maximally effective. Hopefully, the implications emerging from this project, combined with ongoing 
collaborative research and practice in multiple local and global settings, 22 will help us all assess international, 
multicultural, and transformative learning - and apply our insights and understandings - with care, capacity, 
and commitment in the months and years ahead. 
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